Monthly Archives: January 2020

2020-01-08
Configuration Management, Linux, Technology

Make Ansible Nice and Tidy

This article is to demonstrate a simple user administration, and following basic rules:

  • data must be separated from code, avoid data in too many places,
  • critical information must be encrypted,
  • data could be grouped using some environment fact, for example: dev, staging and prod
  • ansible skill is no longer required, once systems is written down.

In an example below, we can forward/filter variable and pass authorized/public key. Sometimes it’s good to keep simple control to what is going to happen to user module/role.

# ansible-galaxy init roles/users --offline
- Role users was created successfully

# cat roles/users/tasks/main.yml
---
# tasks file for users
- name: ensure user exists
  user:
    name: "{{ user.username }}"
    comment: "{{ user.name }}"
    state: "{{ user.state if user_state is defined else 'present' }}"
    password: "{{ user.password }}"
  loop: "{{ users }}"
  loop_control:
    loop_var: user
- name: add authorized key
  authorized_key: user="{{ user.username }}" key="{{ user.publickey }}" state=present
  loop: "{{ users }}"
  loop_control:
    loop_var: user

We can use ansible to create sha-512 passwords:

# ansible -m debug -a msg="{{ 'test123'  | password_hash('sha512') }}" localhost
localhost | SUCCESS => {
    "msg": "$6$UnrDMoPPnIDtNT43$nQvXEqvApVTY09clkvrXg/M4B59qpS2yOM18E9luYXiHQUPmis18bpMKiDxNjd7Wl.QWJM3mFm1TxMnhi74M6/"
}
# ansible -m debug -a msg="{{ 'test123'  | password_hash('sha512') }}" localhost
localhost | SUCCESS => {
    "msg": "$6$Xfc.7wW3XdY8.urH$e40tqEmNHUGFFLdchoXui4.kYIidQm6YxztOQxiviWcKLGtIwCmVNLWGQ/YtM5PnPW5J3dHtm4AClB7OHRv6c/"
}

And use existing/any user public key.

# cat ~/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCJLdGtM14KlKlDr5WpapcbQvE4ONEzDclL8MIrdtocrfX+WJye4sx5v9tEOKAvR6ELuCQiETH3fXXAdTVdl1+lBH4c2bEDR2HfPFkLyXNhTCDD4TkuLUUdsaM44JQWq0O91Enc9zJ4kmcihJ1pGagg3LHfK8tvUzlNSCZgTnEFHNZ7Ir1e16B34TBo67FJC2KYhYQdcH4Osgbcp+1ovenG2He4as/uQogMEqAdx3bZpK/jbiRseHKVdEpSaLPOu6YMmRruoujmHvqHXNbioN8STnvSNQDa88LNRjBJLIl2GyuwR6fxlnWPeXwPRC5aTnlwBs2+o3ON9PU3kpRcxBwDH+FXzazaqEh+Y4oOiytsruJP65NaXTeWWO8f3r55+C5xy7ZsWK2YHet8InXnFAbemFQCwjAWWZ7/+d/qpNdrTzXdJFp4IuwYp+hSSxfC2eqykLHEpAX+D+tL3T75oO1ZGVdlfsplFz5CbY1jNadN7QWJOwMNxAJqiDevRt2+NE= root@VM-101932768

The Ansible recommended structure is using groups and multiple inventories, just like here. Per recommendation, if there’s any common user, we then use a symbolic link.

# tree environments/
environments/
├── dev
│   ├── group_vars
│   │   ├── all
│   │   │   └── users.yml
│   │   ├── db
│   │   └── web
│   └── hosts
├── prod
└── stage

We can group user variables using domain name.

# ansible-vault create dev/group_vars/all/users.yml
# ansible-vault edit dev/group_vars/all/users.yml

---
users:
  - name: Aimee
    username: aimee
    password: $6$UnrDMoPPnIDtNT43$nQvXEqvApVTY09clkvrXg/M4B59qpS2yOM18E9luYXiHQUPmis18bpMKiDxNjd7Wl.QWJM3mFm1TxMnhi74M6/
    publickey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCJLdGtM14KlKlDr5WpapcbQvE4ONEzDclL8MIrdtocrfX+WJye4sx5v9tEOKAvR6ELuCQiETH3fXXAdTVdl1+lBH4c2bEDR2HfPFkLyXNhTCDD4TkuLUUdsaM44JQWq0O91Enc9zJ4kmcihJ1pGagg3LHfK8tvUzlNSCZgTnEFHNZ7Ir1e16B34TBo67FJC2KYhYQdcH4Osgbcp+1ovenG2He4as/uQogMEqAdx3bZpK/jbiRseHKVdEpSaLPOu6YMmRruoujmHvqHXNbioN8STnvSNQDa88LNRjBJLIl2GyuwR6fxlnWPeXwPRC5aTnlwBs2+o3ON9PU3kpRcxBwDH+FXzazaqEh+Y4oOiytsruJP65NaXTeWWO8f3r55+C5xy7ZsWK2YHet8InXnFAbemFQCwjAWWZ7/+d/qpNdrTzXdJFp4IuwYp+hSSxfC2eqykLHEpAX+D+tL3T75oO1ZGVdlfsplFz5CbY1jNadN7QWJOwMNxAJqiDevRt2+NE= root@VM-101932768
  - name: Beta
    username: beta
    password: $6$Xfc.7wW3XdY8.urH$e40tqEmNHUGFFLdchoXui4.kYIidQm6YxztOQxiviWcKLGtIwCmVNLWGQ/YtM5PnPW5J3dHtm4AClB7OHRv6c/
    publickey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCJLdGtM14KlKlDr5WpapcbQvE4ONEzDclL8MIrdtocrfX+WJye4sx5v9tEOKAvR6ELuCQiETH3fXXAdTVdl1+lBH4c2bEDR2HfPFkLyXNhTCDD4TkuLUUdsaM44JQWq0O91Enc9zJ4kmcihJ1pGagg3LHfK8tvUzlNSCZgTnEFHNZ7Ir1e16B34TBo67FJC2KYhYQdcH4Osgbcp+1ovenG2He4as/uQogMEqAdx3bZpK/jbiRseHKVdEpSaLPOu6YMmRruoujmHvqHXNbioN8STnvSNQDa88LNRjBJLIl2GyuwR6fxlnWPeXwPRC5aTnlwBs2+o3ON9PU3kpRcxBwDH+FXzazaqEh+Y4oOiytsruJP65NaXTeWWO8f3r55+C5xy7ZsWK2YHet8InXnFAbemFQCwjAWWZ7/+d/qpNdrTzXdJFp4IuwYp+hSSxfC2eqykLHEpAX+D+tL3T75oO1ZGVdlfsplFz5CbY1jNadN7QWJOwMNxAJqiDevRt2+NE= root@VM-101932768`

Once all setup well, all of our user variables are very simple like above, and our main yml is very simple, nice and tidy like below. The advantages of it is, if there is any change to variable (in this example is user), we don’t need ansible skill to re-code. It just needs some basic yml variable editing skill, and it will reduce human error significantly. It also won’t leave data everywhere.

# cat mainuser.yml
---
- name: run on all host
  hosts: "*"
  roles:
    - users

We can make dev environment for default with defining default in ansible.cfg to avoid running in stage/prod unnecessarily.

[defaults]
inventory = ./environments/dev

And finally running it:

# ansible all --list-hosts
  hosts (1):
    192.168.2.101

# ansible-playbook mainuser.yml --ask-vault-pass
Vault password:

PLAY [run on all host] *************************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************************
ok: [192.168.2.101]

TASK [users : ensure user exists] ***************************************************************************************************************************************************
changed: [192.168.2.101] => (item={'name': 'Aimee', 'username': 'aimee', 'password': '$6$UnrDMoPPnIDtNT43$nQvXEqvApVTY09clkvrXg/M4B59qpS2yOM18E9luYXiHQUPmis18bpMKiDxNjd7Wl.QWJM3mFm1TxMnhi74M6/', 'publickey': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCJLdGtM14KlKlDr5WpapcbQvE4ONEzDclL8MIrdtocrfX+WJye4sx5v9tEOKAvR6ELuCQiETH3fXXAdTVdl1+lBH4c2bEDR2HfPFkLyXNhTCDD4TkuLUUdsaM44JQWq0O91Enc9zJ4kmcihJ1pGagg3LHfK8tvUzlNSCZgTnEFHNZ7Ir1e16B34TBo67FJC2KYhYQdcH4Osgbcp+1ovenG2He4as/uQogMEqAdx3bZpK/jbiRseHKVdEpSaLPOu6YMmRruoujmHvqHXNbioN8STnvSNQDa88LNRjBJLIl2GyuwR6fxlnWPeXwPRC5aTnlwBs2+o3ON9PU3kpRcxBwDH+FXzazaqEh+Y4oOiytsruJP65NaXTeWWO8f3r55+C5xy7ZsWK2YHet8InXnFAbemFQCwjAWWZ7/+d/qpNdrTzXdJFp4IuwYp+hSSxfC2eqykLHEpAX+D+tL3T75oO1ZGVdlfsplFz5CbY1jNadN7QWJOwMNxAJqiDevRt2+NE= root@VM-101932768'})
changed: [192.168.2.101] => (item={'name': 'Beta', 'username': 'beta', 'password': '$6$Xfc.7wW3XdY8.urH$e40tqEmNHUGFFLdchoXui4.kYIidQm6YxztOQxiviWcKLGtIwCmVNLWGQ/YtM5PnPW5J3dHtm4AClB7OHRv6c/', 'publickey': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCJLdGtM14KlKlDr5WpapcbQvE4ONEzDclL8MIrdtocrfX+WJye4sx5v9tEOKAvR6ELuCQiETH3fXXAdTVdl1+lBH4c2bEDR2HfPFkLyXNhTCDD4TkuLUUdsaM44JQWq0O91Enc9zJ4kmcihJ1pGagg3LHfK8tvUzlNSCZgTnEFHNZ7Ir1e16B34TBo67FJC2KYhYQdcH4Osgbcp+1ovenG2He4as/uQogMEqAdx3bZpK/jbiRseHKVdEpSaLPOu6YMmRruoujmHvqHXNbioN8STnvSNQDa88LNRjBJLIl2GyuwR6fxlnWPeXwPRC5aTnlwBs2+o3ON9PU3kpRcxBwDH+FXzazaqEh+Y4oOiytsruJP65NaXTeWWO8f3r55+C5xy7ZsWK2YHet8InXnFAbemFQCwjAWWZ7/+d/qpNdrTzXdJFp4IuwYp+hSSxfC2eqykLHEpAX+D+tL3T75oO1ZGVdlfsplFz5CbY1jNadN7QWJOwMNxAJqiDevRt2+NE= root@VM-101932768`'})

TASK [users : add authorized key] **************************************************************************************************************************************************
changed: [192.168.2.101] => (item={'name': 'Aimee', 'username': 'aimee', 'password': '$6$UnrDMoPPnIDtNT43$nQvXEqvApVTY09clkvrXg/M4B59qpS2yOM18E9luYXiHQUPmis18bpMKiDxNjd7Wl.QWJM3mFm1TxMnhi74M6/', 'publickey': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCJLdGtM14KlKlDr5WpapcbQvE4ONEzDclL8MIrdtocrfX+WJye4sx5v9tEOKAvR6ELuCQiETH3fXXAdTVdl1+lBH4c2bEDR2HfPFkLyXNhTCDD4TkuLUUdsaM44JQWq0O91Enc9zJ4kmcihJ1pGagg3LHfK8tvUzlNSCZgTnEFHNZ7Ir1e16B34TBo67FJC2KYhYQdcH4Osgbcp+1ovenG2He4as/uQogMEqAdx3bZpK/jbiRseHKVdEpSaLPOu6YMmRruoujmHvqHXNbioN8STnvSNQDa88LNRjBJLIl2GyuwR6fxlnWPeXwPRC5aTnlwBs2+o3ON9PU3kpRcxBwDH+FXzazaqEh+Y4oOiytsruJP65NaXTeWWO8f3r55+C5xy7ZsWK2YHet8InXnFAbemFQCwjAWWZ7/+d/qpNdrTzXdJFp4IuwYp+hSSxfC2eqykLHEpAX+D+tL3T75oO1ZGVdlfsplFz5CbY1jNadN7QWJOwMNxAJqiDevRt2+NE= root@VM-101932768'})
changed: [192.168.2.101] => (item={'name': 'Beta', 'username': 'beta', 'password': '$6$Xfc.7wW3XdY8.urH$e40tqEmNHUGFFLdchoXui4.kYIidQm6YxztOQxiviWcKLGtIwCmVNLWGQ/YtM5PnPW5J3dHtm4AClB7OHRv6c/', 'publickey': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCJLdGtM14KlKlDr5WpapcbQvE4ONEzDclL8MIrdtocrfX+WJye4sx5v9tEOKAvR6ELuCQiETH3fXXAdTVdl1+lBH4c2bEDR2HfPFkLyXNhTCDD4TkuLUUdsaM44JQWq0O91Enc9zJ4kmcihJ1pGagg3LHfK8tvUzlNSCZgTnEFHNZ7Ir1e16B34TBo67FJC2KYhYQdcH4Osgbcp+1ovenG2He4as/uQogMEqAdx3bZpK/jbiRseHKVdEpSaLPOu6YMmRruoujmHvqHXNbioN8STnvSNQDa88LNRjBJLIl2GyuwR6fxlnWPeXwPRC5aTnlwBs2+o3ON9PU3kpRcxBwDH+FXzazaqEh+Y4oOiytsruJP65NaXTeWWO8f3r55+C5xy7ZsWK2YHet8InXnFAbemFQCwjAWWZ7/+d/qpNdrTzXdJFp4IuwYp+hSSxfC2eqykLHEpAX+D+tL3T75oO1ZGVdlfsplFz5CbY1jNadN7QWJOwMNxAJqiDevRt2+NE= root@VM-101932768`'})

PLAY RECAP *************************************************************************************************************************************************************************
192.168.2.101              : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
2020-01-04
Configuration Management, Linux, Technology

Ansible Hiera

We might be aware of Puppet Hiera flexibility. Below is an example to proxy hiera in native Ansible (without third party product/plugin).

An example, we have one external host inventory running on RedHat.

# cat demo
[web]
192.168.2.101

Below is the playbook. Once it connects to the inventory host, it will return ansible_facts[‘distribution’] = ‘RedHat’, similar to facter in Puppet. Using this O/S type, we can then define all O/S related roles specific variables in “vars/os_RedHat” variable file.

# cat main.yml
---
- name: talk to all hosts just so we can learn about them
  hosts: all
  vars_files:
    - "vars/os_{{ ansible_facts['distribution'] }}"
  roles:
    - sshd
  tasks:
    - name: tell us the variable in main.yml
      debug:
        var: sshd_ssh_packages

Below are variables that we defined in vars/os_RedHat variable file. By defining O/S specific variables in data hierarchy, the role can be made independent from O/S or any other hierarchy.

# cat vars/os_RedHat
# Information for the sshd for RedHat
sshd_package: "sshd"
sshd_ssh_packages:
  - openssh-server
  - openssh

The issue with Ansible hierarchy is their precedence order couldn’t be easily modified like in Puppet. The precedence order of role and host variables from highest is:

  • role vars
  • hosts vars
  • role default

Unfortunately role variable has higher precedence, therefore the nicest way to define variable is in role default, not in role vars. This role default can then be overwritten with host variables. 

# cat roles/sshd/vars/main.yml
---

# cat roles/sshd/defaults/main.yml
---
sshd_package: "nothing"
sshd_ssh_packages: "nothing"

# cat roles/sshd/tasks/main.yml
---
- name: tell us the variable inside sshd role
  debug:
    var: sshd_ssh_packages

Let’s run the playbook.

# ansible-playbook -i demo main.yml

PLAY [talk to all hosts just so we can learn about them] **********

TASK [Gathering Facts] **********
ok: [192.168.2.101]

TASK [sshd : tell us the variable inside sshd role] **********
ok: [192.168.2.101] => {
    "sshd_ssh_packages": [
        "openssh-server",
        "openssh"
    ]
}

TASK [tell us the variable in main.yml] **********
ok: [192.168.2.101] => {
    "sshd_ssh_packages": [
        "openssh-server",
        "openssh"
    ]
}

PLAY RECAP **********
192.168.2.101              : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

If we comment out host variable, the role variable takes over. It means in case of finding undefined O/S, both host and role can warn/fail task from being processed.

# cat main.yml
---
- name: talk to all hosts just so we can learn about them
  hosts: all
#  vars_files:
#    - "vars/os_{{ ansible_facts['distribution'] }}"
  tasks:
    - name: tell us the variable in main.yml
      debug:
        var: sshd_ssh_packages
  roles:
    - sshd

# ansible-playbook -i demo main.yml

PLAY [talk to all hosts just so we can learn about them] 

TASK [Gathering Facts] 
ok: [192.168.2.101]

TASK [sshd : tell us the variable inside sshd role] 
ok: [192.168.2.101] => {
    "sshd_ssh_packages": "nothing"
}

TASK [tell us the variable in main.yml] 
ok: [192.168.2.101] => {
    "sshd_ssh_packages": "nothing"
}

PLAY RECAP 
192.168.2.101              : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

With ability to separate data and code completely, we can build similar data hierarchy like in Puppet Hiera. However, unfortunately in Ansible, we have to follow fixed Ansible precendence order and work from there.