Using AKS to Build a Developer-First Internal Platform

This article marks the second instalment in our series on building Internal Developer Platforms (IDPs) atop managed Kubernetes services. Having thoroughly explored the Google Cloud ecosystem with Google Kubernetes Engine (GKE) in our first piece, we now pivot to Microsoft’s cloud. In this article, we delve into constructing an IDP on Azure Kubernetes Service (AKS), examining its native tooling and where Crossplane can be strategically employed to bridge gaps in a fully Kubernetes-native approach, setting the stage for a final comparison across hyperscalers.

---

 

Forge Your IDP on AKS: An Azure-Native Blueprint

 

Crafting a Secure and Agile Developer Platform on Azure Kubernetes Service

As with our discussion on GKE, the overarching objective for an IDP remains consistent: empowering platform engineers and security specialists with the most straightforward, rapid, stable, and secure means to deploy and manage applications and their underlying infrastructure. On AKS, this foundational layer is deeply rooted in Azure’s extensive suite of native services, shaping a uniquely Azure-flavoured approach to the IDP blueprint.

 

The Unwavering Principle: Git as the Single Source of Truth

As established in the first article, the immutable principle of Git as the single source of truth underpins every robust IDP. Every application definition, infrastructure blueprint, and cluster configuration — often sculpted with Helm charts or Kustomize overlays — finds its definitive home within a Git repository. This commitment ensures an impeccable audit trail, predictable deployments, and consistent, version-controlled states.If it’s not in Git, it’s basically just a rumour. And we all know how reliable those are.

 

Streamlined Deployments with Azure Arc-enabled GitOps (Flux)

Where GKE leverages Config Sync for its GitOps capabilities, the Azure ecosystem provides Azure Arc-enabled GitOps, powered by Flux v2, to bring your desired state from Git to life on your AKS clusters. This potent solution continuously synchronises your cluster’s actual state with its declared state in Git by leveraging a Kubernetes-native FluxConfiguration Custom Resource. This means your entire GitOps setup is defined and managed directly within Kubernetes YAML, allowing for seamless integration into your declarative workflows.

Azure Arc-enabled GitOps isn’t just for application rollouts; it’s the bedrock for managing:

• Application Lifecycle: Guarantees your microservices always reflect their Git-defined state.
• Cluster Hygiene: Enforces uniform Role-Based Access Control (RBAC), Network Policies, and Resource Quotas across all your clusters.
• Tenant Isolation: Constructs segregated namespaces for diverse teams or environments, each meticulously configured.

With Azure Arc-enabled GitOps, you’ll benefit from excellent stability and a clear lineage of changes, as any deviation from your Git-controlled blueprint is swiftly reconciled. We’ve all had a partner or housemate who won’t rest until the cushions are constantly fluffed and facing the right way, or the coffee table magazines are correctly fanned.

 

Empowering Infrastructure Provisioning with Azure Service Operator (ASO)

In contrast to GKE’s Config Controller (powered by Config Connector), the Azure landscape for Kubernetes-native infrastructure provisioning is championed by the Azure Service Operator (ASO). ASO is a Kubernetes operator that transforms your AKS cluster into a control plane for a vast array of Azure resources. It allows you to provision and manage services like Azure SQL Databases, Storage Accounts, and Managed Identities directly through familiar Kubernetes manifests.

ASO extends your Kubernetes API, enabling developers to express their infrastructure needs in standard YAML, while platform teams maintain firm oversight of the underlying Azure subscriptions and resource groups. This provides a consistent Kubernetes-native interface for managing both applications and their dependent Azure services.

ASO empowers platform teams to:

• Standardise Service Consumption: Curate a gallery of approved Azure services, exposed as Custom Resources within Kubernetes.
• Govern Resource Creation: Developers self-serve infrastructure through declarative Kubernetes manifests, with platform teams enforcing where and how these resources are provisioned.
• Integrate with GitOps Flows: Seamlessly blend ASO with Azure Arc-enabled GitOps (Flux) to ensure your infrastructure provisioning is also driven entirely by Git.

This fusion accelerates development whilst preserving robust governance and control for your security and platform engineers.

 

Bolstering Security and Compliance with Azure Policy via Crossplane

Security and compliance, as we’ve iterated throughout this series, are intrinsically woven into the fabric of your platform. On AKS, Azure Policy is the sentinel of your IDP for both in-cluster and Azure-wide enforcement. Azure Policy integrates with Gatekeeper (an OPA project) for governing Kubernetes configurations directly on your cluster, akin to GKE’s Policy Controller.

However, when it comes to managing Azure Policy Assignments themselves — the overarching configurations that dictate which policies apply to which scopes (e.g., subscriptions, resource groups) — ASO doesn’t support what we need. Therefore, Crossplane with its Azure Provider proves invaluable. While Azure Policy can be assigned via ARM templates or Bicep, Crossplane brings the entire lifecycle of these policy assignments directly under your Kubernetes control plane, aligning perfectly with our GitOps methodology and consistent configuration model paradigm.

Azure Policy, working in tandem with Gatekeeper for AKS and managed by Crossplane, enables you to:

• Enforce Cluster Best Practices: Ensure your Kubernetes configurations adhere to organisational standards.
• Image Governance: Mandate that only pre-approved container images from trusted Azure Container Registry (ACR) repositories are deployed.
• Resource Guardrails: Implement constraints on resource consumption, preventing sprawl and ensuring fair usage across namespaces.
• Network Segmentation: Define and enforce inter-service communication rules to isolate workloads securely.
• Custom Rule Definition: Craft bespoke policies to meet unique organisational security and compliance requirements.

By managing both in-cluster policies via Azure Policy (Gatekeeper integration) and Azure-level Policy Assignments with Crossplane, every tweak to your security posture is version-controlled in Git, passes through familiar pull request workflows, and is automatically reconciled by Kubernetes, thus significantly mitigating risk and bolstering stability.

 

Centralised Secret Management with Azure Key Vault CSI Driver

Just as GKE integrates with Google Secret Manager for credential handling, AKS employs the Azure Key Vault Container Storage Interface (CSI) Driver for sensitive information. This is the go-to solution for direct injection of credentials, such as database passwords, into running pods. It enables your AKS workloads to securely access secrets stored in Azure Key Vault by dynamically mounting them into your pods as a volume. This crucial detail means secrets are never persisted as native Kubernetes Secret objects, drastically reducing their exposure.

 

Advantages of the Azure Key Vault CSI Driver:

• Elevated Security: Secrets are fetched directly from Key Vault at runtime, bypassing storage in the cluster’s etcd or your Git repository.
• Agile Updates: Rotated secrets in Key Vault can be automatically refreshed within running pods, minimising downtime during credential changes.
• Granular Access: Access is meticulously controlled via Azure Role-Based Access Control (RBAC) and Managed Identities, aligning with the principle of least privilege.

 

Integrated Observability

Observability remains a bedrock of any resilient IDP. AKS seamlessly integrates with Azure Monitor for comprehensive metrics, logs, and tracing capabilities, mirroring the integration GKE has with Google Cloud Operations Suite. Many teams also opt for battle-tested open-source stacks like Prometheus and Grafana for metrics, Loki or Fluent Bit for logs, and Tempo or Jaeger for tracing. Instrumentation and dashboard creation can be templated, ensuring every new service comes with default alerts, logs, and runtime metrics straight out of the box for swift diagnostics.

 

Crafting Environments and Workflows for Control and Velocity

AKS’s inherent flexibility supports diverse environment strategies, whether isolated namespaces within a single cluster or distinct clusters for varying criticality levels. With configurations managed in Git and deployed via Azure Arc-enabled GitOps, applying environment-specific settings through overlays or Helm value files is a straightforward affair.

This meticulously engineered workflow dictates that developers submit changes via pull requests to Git. Azure Arc-enabled GitOps then deploys these changes. Azure Service Operator provisions any necessary cloud infrastructure, Crossplane applies the overarching Azure Policies, and the Azure Key Vault CSI Driver securely injects sensitive credentials at runtime. This cohesive approach significantly boosts both developer velocity and the operational control cherished by platform and security teams.

A Real-World Blueprint: The MedTech Transition

Consider a mid-sized MedTech company embarking on a microservices transformation, with a keen eye on rapid, secure, and stable deployments on Azure.

Their platform team’s core mission is to empower engineers to deploy and manage services and infrastructure autonomously, all while operating within robust guardrails. They designate Git repositories as the definitive source for every application and infrastructure configuration.

• Azure Arc-enabled GitOps (Flux) is configured on their AKS clusters, vigilantly monitoring these repositories to ensure that all cluster configurations and application deployments consistently match the Git-defined state, directly from a Kubernetes-native FluxConfiguration.
• When a developer requires a new backend service, such as an Azure SQL Database, they define it using an Azure Service Operator Custom Resource within their application’s infrastructure repository. ASO detects this and provisions the database directly within Azure, simultaneously setting up the required Managed Identity for the application’s secure access.
• Crucially, for database passwords and other secrets, the application pods leverage the Azure Key Vault CSI Driver. Developers simply configure a Kubernetes SecretProviderClass that references the Azure Key Vault secret. Their application’s Kubernetes Service Account, empowered by Managed Identities for AKS pods, is then granted precise Azure RBAC permissions to access that specific secret (also managed by ASO). The password is then dynamically mounted directly into the pod’s filesystem, never residing in Git or the cluster’s static cluster state.
• Crossplane is configured to manage Azure Policy Assignments across their Azure subscriptions and AKS clusters. This guarantees that stringent rules — for example, allowing only approved container images (enforced by Azure Policy’s Gatekeeper integration), enforcing namespace resource quotas, and applying Kubernetes Network Policies for workload isolation — are consistently maintained, upholding compliance and security.

The outcome is a highly efficient and secure deployment pipeline:

• Engineers gain the autonomy to quickly and reliably deploy applications and provision infrastructure through a familiar Git workflow.
• Platform and security teams maintain rigorous control over policies, governance, and foundational configurations, ensuring compliance whilst reducing operational overhead.

This foundational blueprint on AKS delivers the stability, security, and velocity indispensable for modern application delivery. 🚀

 

Example Manifest Files for the Real-World Scenario (Azure Native with ASO & Crossplane)

Here are example manifest files, combining ASO for Azure infrastructure and Managed Identity, and Crossplane for Azure Policy Assignments. Remember to replace placeholders like YOUR_AZURE_SUBSCRIPTION_ID, YOUR_AZURE_TENANT_ID, YOUR_RESOURCE_GROUP_NAME, YOUR_KEY_VAULT_NAME, and YOUR_MANAGED_IDENTITY_CLIENT_ID.

Please also do not use this example for production. This is very high-level and is just to give an idea of what it could look like.

1. (Prerequisite) Azure Key Vault Secret

You’d first need to create your secret in Azure Key Vault.

# Create an Azure Key Vault
az keyvault create --name "my-app-kv" --resource-group "my-aks-rg" --location "uksouth"
# Set a secret in the Key Vault
az keyvault secret set --vault-name "my-app-kv" --name "db-password" --value "your_strong_db_password_for_aks"

Note: my-app-kv is your Azure Key Vault name, and db-password is the name of the secret within it.

 

2. Azure Arc-enabled GitOps FluxConfiguration as a Kubernetes CR

This Kubernetes manifest defines a FluxConfiguration resource. This YAML would reside directly in your Git repository (e.g., platform-repository/arc-gitops/app-flux-config.yaml), and once applied to your Arc-enabled cluster, the Azure Arc GitOps extension’s controller will reconcile it.

# platform-repository/arc-gitops/app-flux-config.yaml
apiVersion: microsoft.flux/v1alpha1
kind: FluxConfiguration
metadata:
name: app-flux # Name of this Flux configuration
namespace: platform-config # Can be any namespace, but often a dedicated one for the platform team
spec:
# General configuration for this Flux setup
scope: cluster # Apply this configuration at the cluster level
namespace: flux-system # The namespace where Flux components will be installed/managed
# Source configuration: where Flux finds your Kubernetes manifests
sourceKind: GitRepository
gitRepository:
url: https://github.com/your-org/your-app-repo.git # Replace with your application Git repo URL
branch: main # The branch to track
# Optional: If your repo is private, you'd add sshKnownHosts and identity (e.g., private key secret)
# sshKnownHosts: |
# github.com ssh-rsa ...
# <Other host keys>
# https:
# url: https://github.com/your-org/your-app-repo.git
# secretRef: # Refer to a K8s secret containing token or basic auth
# name: flux-git-auth
# Kustomization configuration: what Flux should deploy from the repository
kustomizations:
- name: apps # A name for this kustomization sync
path: "./apps/dev-team-a" # Path within your Git repo to the application manifests
syncInterval: "5m" # How often Flux should check for changes
prune: true # Enable pruning to remove resources no longer in Git
timeout: "3m" # Timeout for reconciliation

 

3. Application Deployment (Using Azure Key Vault CSI Driver)

This manifest defines a simple Nginx deployment that will get its secrets via the Azure Key Vault CSI driver. This would reside in your application’s Git repository (e.g., app-repository/apps/dev-team-a/nginx-deployment.yaml), pulled by Flux.

# app-repository/apps/dev-team-a/nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
namespace: dev-team-a # Ensure this namespace exists
labels:
app: my-nginx
spec:
replicas: 2
selector:
matchLabels:
app: my-nginx
template:
metadata:
labels:
app: my-nginx
spec:
serviceAccountName: my-app-aks-sa # Kubernetes Service Account for Workload Identity
containers:
- name: nginx
image: mcr.microsoft.com/oss/nginx/nginx:1.23.3 # Example image
ports:
- containerPort: 80
volumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets-store" # Mount point for secrets
readOnly: true
env:
- name: DB_PASSWORD_FILE_PATH # Your app reads the password from this file
value: "/mnt/secrets-store/db-password"
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: my-db-secret-provider # Link to the SecretProviderClass

 

4. Kubernetes SecretProviderClass for Azure Key Vault (Managed by Flux/GitOps)

This resource defines which Azure Key Vault secrets your pods will access. This would be alongside your application deployment manifests in Git (e.g., app-repository/apps/dev-team-a/secret-provider-class.yaml).

# app-repository/apps/dev-team-a/secret-provider-class.yaml
apiVersion: secrets-store.csi.k8s.io/v1
kind: SecretProviderClass
metadata:
name: my-db-secret-provider
namespace: dev-team-a
spec:
provider: azure
parameters:
usePodIdentity: "false" # Use Workload Identity (true if using old pod identity)
userAssignedIdentityID: "YOUR_MANAGED_IDENTITY_CLIENT_ID" # Client ID of my-app-aks-mi
keyvaultName: "my-app-kv" # Your Azure Key Vault name
objects: |
array:
- |
objectName: db-password
objectType: secret
objectVersion: "" # Use latest version
path: db-password # This is the filename the secret will be mounted as
tenantId: "YOUR_AZURE_TENANT_ID" # Your Azure Tenant ID

 

Important: You will need to obtain the Client ID of the User-Assigned Managed Identity after ASO creates it.

5. Azure SQL Database Provisioning (Azure Service Operator ASO)

This ASO Custom Resource will provision an Azure SQL Database. This manifest would be in your infrastructure Git repository (e.g., infra-repository/sql-db.yaml), managed by Flux.

# infra-repository/sql-db.yaml
apiVersion: dbforazure.azure.com/v1api20230301preview
kind: SqlServer
metadata:
name: my-app-sql-server # Name of the Azure SQL Server
namespace: dev-team-a # ASO resources often live in the same namespace or a dedicated infra namespace
spec:
owner:
resourceGroupName: YOUR_RESOURCE_GROUP_NAME # The Azure Resource Group for the SQL Server
location: uksouth # Your Azure region
administratorLogin: sqladminuser # Admin login for the SQL Server
administratorLoginPassword:
# ASO can fetch admin password from Azure Key Vault or create a K8s secret for it
# For simplicity, we'll assume manual management of this admin password,
# or you'd use ASO's Secret management capabilities here.
# The application user password is handled by Key Vault CSI Driver.
secret:
name: sql-admin-password # Name of a K8s secret containing the admin password
key: password
version: "12.0" # SQL Server version
---
apiVersion: dbforazure.azure.com/v1api20230301preview
kind: SqlDatabase
metadata:
name: myappdb # Name of the database within the SQL Server
namespace: dev-team-a
spec:
owner:
group: dbforazure.azure.com
kind: SqlServer
name: my-app-sql-server # Refers to the SqlServer created above
collation: "SQL_Latin1_General_CP1_CI_AS"
sku:
name: "Standard"
tier: "Standard"
capacity: 10 # DTUs

 

6. Azure User-Assigned Managed Identity (ASO UserAssignedIdentity Custom Resource)

This creates the User-Assigned Managed Identity in Azure, managed by ASO. This would be in your platform-repository/azure-iam/ directory, managed by Flux.

# platform-repository/azure-iam/my-app-managed-identity.yaml
apiVersion: managedidentity.azure.com/v1api20220131preview
kind: UserAssignedIdentity
metadata:
name: my-app-aks-mi # Name for the Managed Identity resource in Kubernetes (and in Azure)
namespace: dev-team-a # Namespace where the Managed Identity CR lives
spec:
owner:
resourceGroupName: YOUR_RESOURCE_GROUP_NAME # The Azure Resource Group where the MI will live
location: uksouth # Azure region for the Managed Identity

 

7. Azure RBAC Role Assignment (ASO RoleAssignment Custom Resource)

This grants the Managed Identity permissions to access your Azure Key Vault. This would also be in your platform-repository/azure-iam/ directory, managed by Flux.

# platform-repository/azure-iam/my-app-kv-access-role-assignment.yaml
apiVersion: authorization.azure.com/v1api20220401
kind: RoleAssignment
metadata:
name: my-app-kv-secret-reader-role-assignment # Name for the Role Assignment in Azure
namespace: dev-team-a # Namespace where the RoleAssignment CR lives
spec:
owner:
group: keyvault.azure.com
kind: Vault
name: my-app-kv # The Azure Key Vault name
# You might need to specify the resource group if not inherited or in a different RG
# resourceGroupName: YOUR_RESOURCE_GROUP_NAME
roleDefinitionId: "/subscriptions/YOUR_AZURE_SUBSCRIPTION_ID/providers/Microsoft.Authorization/roleDefinitions/21090740-5e88-453d-9d55-2478470b1d3d" # Built-in "Key Vault Secrets User" role ID
principalId:
from:
name: my-app-aks-mi # The name of your UserAssignedIdentity CR (managed by ASO)
kind: UserAssignedIdentity
group: managedidentity.azure.com

 

8. Kubernetes Service Account (Manually Populated Client ID for simplicity)

This is your Kubernetes Service Account for the application pod.

# app-repository/apps/dev-team-a/my-app-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-app-aks-sa
namespace: dev-team-a
annotations:
# This annotation links the Kubernetes SA to an Azure Managed Identity for Workload Identity
# IMPORTANT: You need to manually populate YOUR_MANAGED_IDENTITY_CLIENT_ID here
# after ASO creates the UserAssignedIdentity and its clientId becomes available in its status.
# For full automation, a templating solution (e.g., Kustomize replacements, Helm lookup)
# would dynamically inject this from the ASO UserAssignedIdentity's status.
azure.workload.identity/client-id: "YOUR_MANAGED_IDENTITY_CLIENT_ID"

 

9. Azure Policy Assignment (Crossplane PolicyAssignment Custom Resource)

This manifest assigns an Azure Policy using Crossplane. This would be in your platform-repository/azure-policies/ directory, managed by Flux.

# platform-repository/azure-policies/aks-allowed-images-policy-assignment.yaml
apiVersion: policy.azure.upbound.io/v1beta1
kind: PolicyAssignment
metadata:
name: aks-allowed-images-policy # Name for the Crossplane PolicyAssignment CR
namespace: crossplane-system # Or a dedicated namespace for Crossplane-managed Azure resources
spec:
forProvider:
scope: /subscriptions/YOUR_AZURE_SUBSCRIPTION_ID/resourceGroups/YOUR_RESOURCE_GROUP_NAME # Scope of the policy assignment
policyDefinitionId: /providers/Microsoft.Authorization/policyDefinitions/a08ea5c6-2d01-4475-8164-11d731e0b522 # Built-in policy for allowed images
parameters: # Parameters for the policy definition
allowedImageRegex:
value: "^(mcr.microsoft.com|youracr.azurecr.io)/"
exclusions:
value:
- "kube-system"
- "gatekeeper-system"
- "azure-arc"
- "aso-system" # Add aso-system
- "crossplane-system" # Add crossplane-system
- "flux-system" # Add flux-system
enforcementMode: Default # Or "DoNotEnforce" for audit-only
providerConfigRef:
name: azure-provider-config # Reference to your configured Crossplane Azure ProviderConfig

 

Final Thoughts

AKS, when combined with Azure-native tooling and GitOps workflows, provides a powerful foundation for building secure, scalable Internal Developer Platforms. By integrating services like Azure Arc-enabled GitOps, ASO, Crossplane, and Key Vault CSI, platform teams can enforce governance and security while giving developers the autonomy to move fast.

Compared to GKE, AKS stands out for its enterprise-grade policy control and deep integration with Azure’s identity and security stack. With the right patterns in place, AKS becomes more than a Kubernetes service — it becomes a developer-first platform that balances speed with control.

Stay tuned for the final instalment, where we’ll wrap up with a comprehensive look at building Internal Developer Platforms on Amazon EKS.

---

Want to accelerate your IDP journey? Mesoform helps teams design, build, and operate production-grade platforms on any cloud, without reinventing the wheel. Reach out to see how we can help.