19. 09. 2024
Development, DevOps
19. 12. 2024
Luigi Miazzo
Automation, Development, DevOps
Embracing Idempotency: Writing Your Own Ansible Collection – From Code to Tests
Ansible is a powerful automation tool that simplifies the configuration, deployment, and management of systems. At its heart lies the concept of idempotency — the guarantee that applying the same operation any number of times will yield the same result. Writing your own Ansible collection can unlock a new level of customization and control for your automation needs, tailored to your unique use cases, using a very powerful, yet simple language: python. This blog post will guide you through creating a custom Ansible collection, structuring your library, and ensuring robust functionality through testing.
Why Write a Custom Ansible Collection?
While Ansible provides a wealth of built-in modules and a rich ecosystem of community packages, there are scenarios where custom functionality is necessary. Here are some reasons to consider writing your own library:
- Complex Business Logic: Your environment may demand operations not covered by Ansible’s default modules or may require the customization of user-specific features to align with unique workflows
- Optimization: Custom modules can improve efficiency by reducing unnecessary steps or simplifying complex tasks
- Reusability: Centralizing logic in a library ensures consistent usage across your playbooks
Setting Up Your Development Environment
Before diving into code, set up your development environment:
- Install Ansible: Ensure you have the latest version of Ansible installed
- Python Environment: Use a virtual environment to isolate dependencies
- Editor: Choose an IDE or text editor with good Python support, like NeoVIM or VS-Code
- Libraries: Install Ansible development and testing dependencies:
$pip install ansible-core ansible-lint ansible-test - Create a Collection Environment: Initialize a new Ansible collection using the
ansible-galaxycommand:$ ansible-galaxy collection init my_namespace.my_collection
Anatomy of an Ansible Module
When building a custom module within an Ansible collection, it’s important to understand the broader structure of a collection. Ansible collections organize modules, plugins, and other resources into a standardized layout, which facilitates sharing and reuse. Here’s what a typical collection structure looks like:
$ tree my_namespacemy_namespace/ └── my_collection/ ├── README.md ├── galaxy.yml ├── docs/├── roles/├── plugins/ │ ├── modules/ │ │ └── my_module.py │ └── filter/ ├── module_utils/ │ └── my_utils.py └── tests/ ├── unit/| └── modules/│ └── my_module.py └── integration/
Key Directories and Files:
README.md: Describes the collection and its usagegalaxy.yml: Metadata for the collection, including its name, version, and dependenciesplugins/modules/: The primary location for your custom modulesmodule_utils/: Contains reusable Python utilities that modules can sharetests/: Houses unit and integration tests for your collection
This structure ensures that your modules and plugins are well-organized, maintainable, and compatible with Ansible’s distribution model. If desired, you can build and publish your collection using
$ ansible-galaxy collection build
making it accessible to others within the community.
Crafting the Core: Building an Ansible Module
An Ansible module is essentially a Python script that communicates with Ansible through JSON. Here’s a simple skeleton for a module:
$ cat my_namespace/my_collection/plugins/modules/my_module.py
from ansible.module_utils.basic import AnsibleModule
def main():
module_args = dict(
name=dict(type="str", required=True),
state=dict(type="str", choices=["present", "absent"], default="present")
)
module = AnsibleModule(
argument_spec=module_args,
supports_check_mode=False,
)
result = dict(
changed=False,
message='',
)
name = module.params['name']
state = module.params['state']
if state == "present":
result["message"] = f"Resource {name} is now present."
elif state == "absent":
result["message"] = f"Resource {name} is now absent."
result["changed"] = True
module.exit_json(**result)
if __name__ == '__main__':
main()Key Components:
argument_spec: Defines parameters your module acceptsresult: Provides data to the Ansible context once the module has finishedsupports_check_mode: Indicates whether the module supports dry-run operationsmodule.exit_json(...): Exit properly from the module
Idempotency is king. You may need to communicate with the filesystem or call some subroutines to achieve some changes: here’s where you need to be careful, changing the system’s status only when needed, and always reporting it to the user through the result["changed"] variable. Here’s a brief set of key points to always bear in mind when writing a module:
- Clarity and Simplicity: Write clear, concise and modular code to make the module easier to understand and maintain
- Error Handling: Anticipate potential issues and provide informative error messages through various log levels
- Check Mode Support: Include support for
check_modeto allowdry-runscenarios - Parameter Validation: Always validate user input and system’s status; never assume anything
- Idempotency: Ensure the module only makes changes when necessary by checking the current state before applying operations, rolling back in case of errors, and cleaning up temporary environments
Testing Your Module
Testing ensures your module behaves as expected in various scenarios. Ansible provides tools like ansible-test for this purpose. Briefly, here’s how you can use it:
1. Unit Tests
Unit tests are automated tests that verify individual components in isolation. They are important for detecting early bugs and ensuring that small parts or logic of the application function as expected by mocking up other procedures to make them deterministic. The following is an example with unittest, but you can choose the technology you prefer. We won’t go too deeply into how to write a complete functioning test, just the following code snippet which is enough to give you an idea of what it looks like.
$ cat my_namespace/my_collection/tests/units/module/my_module.py
import unittest
class TestMyModule(unittest.TestCase)
def test_present_state(self):
set_module_args(
{
"name": "my_resource",
"state": "present",
},
)
with self.assertRaises(AnsibleExitJson) as result:
my_module.main()
self.assertTrue(result["changed"])
self.assertEqual(result["message"], "Resource my_resource is now present")Through assertions and mockups, we can effectively test our code in various cases, from the most frequent/probable to the least, raising errors when procedures don’t work as expected in our test cases.
2. Integration Tests
Integration tests on the other hand verify that different components of a system work together as expected. They are important for identifying issues in interactions between different modules or components. In this example, integration tests are represented by playbooks that use your newly created module. Here’s an example:
$ cat my_namespace/my_collection/tests/integration/targets/my_test.yaml
- hosts: localhost
connection: local
tasks:
- name: Test my custom module
my_module:
name: test_resource
state: present
Running Your Tests
Now that we’ve implemented all of our tests, it’s time to run them! But first, a sanity check:
$ ansible-test sanity
This command (ansible-test) is the utility which will help us correctly run our tests. Sanity checks perform a static analysis of our code, ensuring that the target python version we intend to use is compatible with our code (specifiable via the parameter --python <version>, e.g. --python 3.13), enforcing Ansible standards and requirements.
Then we can run all of our unit tests with the command:
$ ansible-test units
We could really spend another post talking about testing environments, but just for basic knowledge, know that ansible-test allows you for instance to run your tests even on Docker containers or to activate only specific tests in case of debugging.
Last but not least, we can run our integration test:
$ ansible-test integration
Using Your Collection
To actually use your custom collection, specify the ANSIBLE_COLLECTIONS_PATH environment variable or include the path in your ansible.cfg. Alternatively, you can package your collection and place it in Ansible’s default collection directory.
Conclusions
Writing your own Ansible library opens up endless possibilities for tailored automation solutions. By embracing idempotency and adhering to best practices, you’ll create robust, reusable modules that integrate seamlessly with Ansible’s ecosystem. We haven’t had the chance to touch on other useful topics like generating custom facts, roles, filters, actions, etc., but I hope this gives you an idea of how vast Ansible can be. Don’t forget to thoroughly test your code to ensure reliability in production environments. Happy automating!
These Solutions are Engineered by Humans
Did you find this article interesting? Are you an “under the hood” kind of person? We’re really big on automation and we’re always looking for people in a similar vein to fill roles like this one as well as other roles here at Würth Phoenix.
Luigi Miazzo
Author
