ericsysmin's DevOps Blog

Configuring Docker Desktop on WSL2

Configuring Docker Desktop on WSL2

First steps, you’ll need to install and configure WSL2. To install WSL2 you can use the Microsoft Store or follow these instructions: https://learn.microsoft.com/en-us/windows/wsl/install

Then to install Docker to run on Windows and WSL2 you’ll need to follow these instructions: https://docs.docker.com/desktop/wsl/

During some testing and trying to simplify my WSL2 environment I stumbled upon an annoying issue that prevented me from running docker ps each time I attempted to run docker ps I’d receive the following error.

To get around this issue you’ll need to run the following commands:

Once those are ran you should be able to run docker without hitting permissions errors.

Those commands are adding/ensuring that the docker group exists and adding your existing user to the docker group. It then modifies the docker.sock to allow the docker group access to the socket.

Continue reading...

How to Install Pyenv on MacOS

Steps to install Pyenv on MacOS

There are a few ways on MacOS to install Python. You can install it via Brew, or by using Pyenv. After needing to switch between different versions of Python often I’ve decided to move to Pyenv. Prior to these steps I removed all versions of Python installed directly with Brew.

1. Update Brew and install prerequisites
We will need to update brew.

In some cases when installing Python >=3.12.1 we will need ncurses. If it’s missing you can install using:

2. Install Pyenv using brew

The recommended way to install pyenv on MacOS is brew.

3. Brew doctor fix

If you want to avoid brew doctor warning about “config” scripts existing outside the system or Homebrew directories please include the following in your shell.

4. Configure your Zsh profile.

If you wish to use Pyenv in non-interactive shells, add the following:

5. Restart shell

6. Install python 3.12

I am going to show how to install python 3.12 but you can select any version of your choice.

7. Switch between your python versions

pyenv shell <version> – modifies python for the current shell session

pyenv local <version> – modifies the python used in the current directory (or subdirectories)

pyenv global <version> – modifies the python used for your user account

 

Continue reading...

How to Install Pyenv on Ubuntu 22.04

Due to the slowness of repositories or even lack thereof being updated with specific versions of Python, I’ve decided to move some of my environments over to Pyenv to allow me to dynamically install and configure Python specifically for my environment. As it turns out this will also allow VS Code to allow me to choose the version of Python that I’d like to use when testing. So, here’s a quick guide to installing Pyenv on Ubuntu 22.04

Steps to install Pyenv on Ubuntu 22.04

1. Update and Install Dependencies

We need to ensure our package cache is updated, and then install the dependencies to download, and build Python from Pyenv.

2. Install Pyenv using pyenv-installer

3. Configure user profile to use pyenv

Ensure the following is in your ~/.bash_profile (if exists), ~/.profile (for login shells), ~/.bashrc (for interactive shells), or ~/.zshrc

Optionally enable pyenv-virtualenv

4. Reload your profile

5. Install python using pyenv

6. Set your python version

pyenv shell <version>  — select just for current shell session
pyenv local <version>  — automatically select whenever you are in the current directory (or its subdirectories)
pyenv global <version>  — select globally for your user account

7. Validate your installation of python

or

 

Continue reading...
Ansible Collections: Automating the Release Process to Galaxy

Ansible Collections: Automating the Release Process to Galaxy

Since we are moving to Ansible Collections, some things are changing. Now when you create your collection and update your collection Ansible Galaxy will no longer automatically discover your collection via a Webhook. Now for Galaxy to know about your collection you have to upload a tar.gz file that containers the result of the ansible-galaxy collection build command.

However, many of us may still want to automate that process, and with @geerlingguy‘s help I was able to fully automate the release process, not from just tagging a release, but creating a release as we would before. So how does this work?

Creating the Build Directory

First, we need to create the build/  directory and include a couple of files.

Instead of having a galaxy.yml file in our root, we will need to generate the file when we execute the playbook.

This is the galaxy_deploy.yml  playbook.

You’ll then need to create a build/templates folder, and create the galaxy.yml.j2  file within the templates folder.

Edit the values to fit your Ansible Collection, the only var I use is {{ tag }}  which will be used later on.

Ok, so now that we’ve created the build components now we need to do the automation part of this. I chose to use GitHub Actions again, as they are the recommended path for the repositories sitting at https://github.com/ansible-collections.

Configuring the GitHub Action Workflow

In your .github/workflows/  folder you’ll need to create a release workflow. To do this I used the following GitHub Actions Workflow YAML. I called it release.yml , and it sits at .github/workflows/release.yml this is an example of what you can use.

Using the on  value we are able to set the workflow to only execute when a release is created in GitHub. This will ensure we have a GitHub ref to be used against the playbook. It will also sync your Ansible Galaxy release with GitHub release actions.

If you noticed we have a key here that provides our Ansible Galaxy token ${{ secrets.ANSIBLE_GALAXY_TOKEN }} for us to use this token we need to get it from Ansible Galaxy, and add it to our repository secrets. You can find your key here https://galaxy.ansible.com/me/preferences under API Key.

Ansible Galaxy Preferences Page

Within the GitHub repo go to the Settings -> Secrets.
GitHub Settings Page

Then when on that page we will add a new secret and name it ANSIBLE_GALAXY_TOKEN

GitHub Secrets Page

Now when the Workflow runs it will grab this secret from your GitHub and be able to authenticate to Ansible Galaxy.

This section tells GitHub Actions to only run this workflow when a release is created. That is done in the GitHub UI, just like you did in the past to release a new version of a role.

This section:

  • checks out the code
  • configures python 3.8 on the host
  • installs the latest version of python pip
  • installs ansible
  • then runs the playbook with the github.ref  value from the GitHub Release action

Once this is done you will have the release version uploaded automatically to your Ansible Galaxy account.

Continue reading...
Ansible Collections: Role Tests with Molecule

Ansible Collections: Role Tests with Molecule

As many of you know or are finding out, Ansible is moving to Collections. But what does that mean? Well, it’s been a long time waiting but Collections provide a way to namespace modules, roles, and playbooks that can all be combined in a single package for you to consume. It also allows businesses, partners, and contributors to update modules without adhering to the Ansible core release cycle. So, if AWS updates their API, then the modules that go with those will be instantly accessible, or at least faster than we used to wait for core releases to get those modules. But what does this mean for roles?

Move to FQCN (Fully Qualified Collection Name)

As many of us our finding out we are needing to move our roles to the collection design. However, now we need to figure out how to test them with the new design and using the Collection Namespaces aka FQCN (Fully Qualified Collection Name) So what we used to write

Will now end up something like this

We also are going to have a new folder structure using ansible_collections/namespace/collection_name

Luckily the molecule team and all of its contributors ensured that collections are recognized and supported. And I will cover how we can test this with GitHub Actions (which happen to also be the preferred way at this moment to test your collections on the official Ansible-Collections Github https://github.com/ansible-collections.

Single Command Testing

Example with multiple role tests in one collection molecule test.

As you see, in this format I had to make sure RedHat specific roles don’t get run on non-RedHat systems. But then this tests every role together and doesn’t easily allow me to scale to even more operating systems.  It will get long, and crazy with lots of when statements, and each time you add a role, you’ll have to edit this one and ensure the environment configuration is correct for all of the roles. It was at this point I realized maybe I should move away from Travis-CI, and also was encouraged to by the Ansible team. Gundalow and others recommended moving to GitHub Actions which are the preferred method now in the community. So I explored that option.

I also at this time decided to move away from the monolithic “default” scenario and instead divide scenarios based on roles. This is what I moved to.

Now I was able to individually ensure each environment was correct. That includes each roles’ dependencies, etc, and that it doesn’t affect the others. It prevented any form of cross-contamination on testing, and what’s expected in requirements. This fixed one of my issues I had with testing, however, it did add a bit of complexity, but each one is almost identical so I could easily copy and paste one of these to help build tests for the new role.

Matrix Testing with GitHub Actions

When I started testing how I would do this using Github Actions I explored using Matrix workflows. That looked something like this (which was really awesome because I couldn’t do it in Travis-CI…I don’t know if I can go back to Travis-CI because apparently for me GitHub tests are much faster.)

However, here’s the issue with this…each commit no matter where it’s made will trigger the matrix to execute meaning 4 x 6 tests! Even though I modified 1 role. Also, EPEL doesn’t work on Ubuntu or Debian. So then I’d have to use a lot of these exclude  statements:

Of course, that’s not scalable. So, I gave it a bit of thought. Why don’t we treat each role for what it is? It’s a separate role. Editing Role1 shouldn’t affect Role2 or even need to test Role2 in this situation. So I decided to create multiple workflows. Using GitHub Workflows I created the following structure.

Each workflow is specific to each role, and each one looks similar. This is the template I used.

You can change the operating systems if you want, this is just one of the examples I had. I solved the issue with the cross-contamination I had earlier, as well as made the tests easier to verify and check, as well as independent test state icons for the README.md. But I still had an issue. If I make a change to Role1, Role2 still builds…not desired and wastes build time against GitHub Actions.

Luckily in GitHub Actions, we can do include, or exclude paths on the trigger. So I replaced this section on: ["push", "pull_request"] with

So now the role only executes when the items specific to that role are edited. Saving me possibly money, and build time so other jobs in my GitHub can execute.

Now my completed .github/workflow/chrony.yml  looks like this:

Since these changes have been made now I am able to ensure that all of my roles are independently tested each time they are edited without treating everything as one giant repo and having tests run for 10+ minutes each as all of my roles execute. Now they are all tested in parallel, and against their own supported operating systems.

To see a copy of the repository used for this you can see https://github.com/ericsysmin/ansible-collection-system which you are free to clone, modify, change, use a reference. I did make changes to the Dockerfile because I do not host my own docker images, and don’t plan to. I highly suggested taking a look at my molecule/role_name/Dockerfile.j2  files to get an idea on what I did to get services to work. My changes to Dockerfile.j2 are based on Multi-distribution Ansible testing with Molecule on Travis-CI, and check out molecule/role_name/molecule.yml  to see how I pass through the parameters.

If you have questions please feel free to comment.

 

Continue reading...