Comparing Configuration Management Tools for Kubernetes: Helm, Kustomize, KPT, and Jsonnet

Configuration management can be a daunting task in Kubernetes. Especially if you are dealing with complex Kubernetes environments and many microservices all over the place.

In those multicomponent environments, it would be unnecessarily cruel to be sentenced to manually editing endless YAML manifest files.

Luckily, configuration management tools like Helm, Kustomize, KPT, and Jsonnet are beacons of hope. But are they up for the task in multicomponent, complex environments with interconnected microservices?

In this article, we’ll share our experiences in exploring various approaches, comparing Kustomize, Helm, KPT, and Jsonnet.

Helm Vs Kustomize: Pros and Cons

Helm: The Kubernetes Package Manager

Helm is the beloved package manager for Kubernetes that simplifies the deployment and management of applications. Helm uses charts, which are collections of YAML templates that describe a set of Kubernetes resources.

Helm uses a templated manifest system where you write general manifests with placeholders for variable parts. You can customize these variables in the values.yaml files and use helm’s tooling to complete the deployment.

Pros:

• Simplifies Manifest Creation: Helm’s templating engine allows for dynamic and reusable configurations.
• Supports Deployments and Upgrades: Helm can manage complex deployment tasks, including database schema migrations, making it easier to handle application updates.

Cons:

• Limited Flexibility: You can only customize what the chart author has exposed as variables.
• Inconsistent Namespace Handling: Helm can be inconsistent with specific naming schemes for namespaces, particularly in Custom Resource Definitions (CRDs), leading to potential deployment issues if not carefully managed.

Kustomize: Kubernetes’ Configuration Tool

Next on our list is Kustomize, the native Kubernetes tool for managing configurations. It uses a declarative approach to manage Kubernetes configurations.

Kustomize uses a base manifest, which is the core YAML configuration that defines your Kubernetes resources. This base manifest is the starting point for all environments.

Customizations for different environments are defined in overlays. Overlays are YAML files that specify changes to be applied to the base manifest. These changes could include modifications to resource specifications, additional labels, or environment-specific configurations.

To deploy your application with Kustomize, you use kubectl with the -k flag, pointing to the appropriate overlay directory.

Pros:

• High Flexibility: Kustomize offers more flexibility than Helm. You can use overlays and patches to modify base resources without being limited to predefined templates or variables.
• Simplified Updates: Customizations typically remain intact when updating the base manifest, allowing core configuration updates without reapplying environment-specific changes, simplifying management across environments.

Cons:

• Incomplete Abstraction: Kustomize can struggle with certain elements, such as namespaces embedded within larger strings, which can cause configurations to break, leading to deployment issues.

Kustomize and Helm: Our Verdict

Both Kustomize and Helm have their strengths and can work well for isolated applications. But they also have significant limitations that can impact productivity and reliability in managing complex Kubernetes environments.

A common issue with both Helm and Kustomize is that you only see the end result after deployment. This means you must deploy the configurations on the actual development platform to verify if they are correct, complicating debugging and validation.

Exploring Alternatives: KPT (Kubernetes Packaging Tool) and Jsonnet

KPT: Configuration Authoring and Automation

KPT is an alternative to Kustomize, emphasizing local customization. You publish a canonical package in a Git repository, which an integrator pulls and customizes locally. This approach allows for manual changes before applying the manifests to the cluster.

Instead of expressing configuration as code or templates that generate the configuration, KPT represents Configuration as Data. Furthermore, KPT leverages the OpenAPI resource schema to enable you to define configurations.

Pros:

• Local customizations: Local rendering of manifests simplifies debugging by allowing inspection before deployment. By enabling local customizations, KPT allows for more complex and specific changes.
• Intelligent analysis: The KPT tool is capable of detecting both declarative and manual changes. When a new version is pulled, it can automatically integrate these manual changes.

Cons:

• Manual changes can be cumbersome: The need for manual changes can complicate management, particularly for large projects.

Jsonnet: The Programming Language for JSON

Jsonnet is not a typical Kubernetes tool but a powerful programming language for generating JSON output, which can be easily converted to YAML for Kubernetes. It offers a more structured and powerful way to transform manifests compared to Helm or Kustomize.

Pros:

• Full Programming Language: Jsonnet’s flexibility allows for any transformation and logic application, offering unmatched customization potential.
• Structured Data Handling: Jsonnet recognizes structured JSON, reducing the risk of generating invalid output.

Cons:

• Additional Tooling Required: Jsonnet needs supplementary tools to integrate fully with the Kubernetes ecosystem.

Also interesting to know: tools like Grafana Tanka and Kapitan are built on top of Jsonnet and provide additional features, such as environment management, a more intuitive syntax, reusable templates and integration with complex deployment workflows.

While Kapitan and Grafana Tanka offer powerful features and great ideas, we didn’t further explore them, because we needed a solution focused on component-level management rather than platform-wide deployment.

KPT and Jsonnect: Our Verdict

Despite needing additional tools for full integration, Jsonnet emerged as the best fit for our needs. It allowed us to specify changes once and apply them consistently, providing flexibility to tailor configurations to our specific requirements.

Conclusion

Choosing the right configuration management tool for Kubernetes depends on your specific needs:

• Helm and Kustomize: These tools offer robust solutions for isolated applications but may struggle with complex, multicomponent platforms.
• KPT: Provides powerful local customization capabilities but can become cumbersome for large projects due to manual management.
• Jsonnet: Offers great flexibility and structured data handling for generating Kubernetes manifests, though it requires additional integration tools.

If you are managing many interconnected microservices and modular architectures, a tailor-made configuration solution based on these existing tools may be the best approach. It can leverage their strengths while addressing the specific needs of your environment.

About Klarrio

At Klarrio, we design cloud native, cloud agnostic software solutions to empower our customers to control their data, limit cloud costs, and optimize performance. We ensure flexibility for scalable platform building across various cloud and on-premises infrastructures, prioritizing privacy, security, and resilience by design.

We are platform pioneers at heart, with a proven track record in building self-service data platforms, Internal Developer Platforms, log aggregation platforms, and other innovative software solutions in various domains: from Telecom, Transportation & Logistics, Manufacturing, Public Sector, Healthcare to Entertainment.

Beyond technology, we actively collaborate and share knowledge, both in-house and together with our customers. True impact is achieved together.