Getting Started with cert-manager

Tiffany Jernigan & Leigh Capili

This guide leverages the cert-manager official documentation. Refer there if you need a deeper dive.

What is cert-manager?

TLS certificates are a popular way for clients and servers to authenticate (prove who they are) to each other. For instance, each time you connect to an address with the https:// protocol, the server (the website) presents to the client (your web browser) its TLS certificate—a bit like an identity document proving “yes, I’m really www.yourbanksite.com, and you’re not about to send your precious banking login information to a malicious person trying to impersonate your banking website.”

There is also mutual TLS (mTLS), when both the client and the server present a certificate to each other. While rarely used in web browsers, this method is frequently used by services that need to authenticate to each other in a secure fashion. For instance, in Kubernetes, each kubelet (the Kubernetes agents that run on every node of the cluster) authenticates to the API server using mTLS. This means that when new nodes join the cluster, Kubernetes needs to issue them their own certificate. Furthermore, since certificates have an expiration date, they need to be renewed regularly. Bottom line: you need a way to automatically issue and renew these certificates.

The Kubernetes API has a CertificateSigningRequest resource to automate certificate issuance and renewal, but currently it is mostly intended for Kubernetes’ internal use. It doesn’t offer a lot of flexibility otherwise. This is where cert-manager shines. It supports numerous ways of generating certificates. It can be used as a standard certificate authority, but it can also use external authorities like HashiCorp Vault, services like Let’s Encrypt, or it can generate self-signed certificates. Even better, all these methods can be supported side by side, so that a single cert-manager install can support everyone’s TLS needs on the cluster.

Another strong point of cert-manager is its deep integration with other Kubernetes resources. For instance, if you need a certificate for a web application exposed through an Ingress resource, you can ask cert-manager to obtain and configure that certificate by adding an annotation to that Ingress resource. Since tools like Carvel, Helm, or Kustomize almost always expose hooks to add annotations to the resources that they provision, it gives us a straightforward method to enable HTTPS on your apps and services without having to fiddle with custom resources, for instance.

Alright, you’re sold on cert-manager. How do you get started with it? In the following tutorial, you’re going to use it together with Contour to expose a web app (Grafana) over HTTPS.

Prerequisites

  • A Kubernetes cluster
  • Docker installed – or have a tool like getent to get an IP address from a load balancer.

Add the package repository with your cluster. To install it, you will need the Carvel tool, kapp-controller.

To install kapp-controller, follow the Getting Started with kapp-controller guide and use the Package Consumption section for the packages you install in this guide.

Install cert-manager

  1. Add the package repository.

    tanzu package repository add tce-repo --url projects.registry.vmware.com/tce/main:stable --namespace tanzu-package-repo-global
    
  2. Verify it is there and has reconciled.

    tanzu package repository list --namespace tanzu-package-repo-global
    
  3. Now you can see all of the available packages.

    tanzu package available list
    
  4. You will need to see what versions of cert-manager are available.

    tanzu package available list cert-manager.community.tanzu.vmware.com
    
  5. Pick the latest version, replace it in the command, and install it.

    tanzu package install cert-manager \
    --package-name cert-manager.community.tanzu.vmware.com \
    --version <VERSION>
    
  6. Now you can see that you have cert-manager installed.

    tanzu package installed list
    

Ingress Controller

The Kubernetes Ingress is an API object that provides routes for traffic (HTTP and HTTPS) from outside the cluster to services within the cluster.

To satisfy an Ingress, you need an Ingress Controller. The repository that was added has a package for the Contour Ingress Controller.

The Grafana package, which you will be installing in the next section, needs an Ingress Controller. Additionally, you will need this Ingress Controller for enabling TLS later.

  1. List the available packages in your repository.

    tanzu package available list contour.community.tanzu.vmware.com
    
  2. Install the latest version of Contour.

    tanzu package install contour \
    --package-name contour.community.tanzu.vmware.com \
    --version <VERSION>
    

Grafana

You will use the Grafana dashboard for your website.

  1. List available packages in your repository.

    tanzu package available list grafana.community.tanzu.vmware.com
    
  2. Install Grafana to get a metrics dashboard. You will configure Grafana to serve HTTPS using a Certificate object.

    tanzu package install grafana \
    --package-name grafana.community.tanzu.vmware.com \
    --version <VERSION>
    
  3. Wait for the load balancer for Grafana to be ready. Run Ctrl-C to terminate the command when the load balancer is ready.

    kubectl -n grafana get service grafana -w
    
  4. Go to the EXTERNAL-IP path in your web browser.

    kubectl -n grafana get service grafana -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
    

    To log in, the username and password are just admin by default. If you click on the set of four squares, you can go to the existing dashboards and see data from Prometheus.

Ingress

First, you need an Ingress. This guide will use nip.io, which provides wildcard DNS for any IP address for the domain. If you’d like to use your own domain, follow the instructions in the guide Installing Harbor on Kubernetes with Project Contour, Cert Manager, and Let’s Encrypt to set up DNS.

  1. If you’re using nip.io (or another similar site), you’ll need to get the IP address for the Envoy load balancer.

    LB=$(kubectl -n projectcontour get svc envoy -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
    echo $LB
    LBIP=$(docker run alpine getent hosts $LB | awk '{print $1}')
    echo $LBIP
    
  2. Now create the Ingress. If you’re using your own domain, you’d substitute it here instead of ${LBIP}.nip.io.

    kubectl -n grafana create ingress grafana --class=contour  "--rule=grafana.${LBIP}.nip.io/*=grafana:80,tls=grafana.${LBIP}.nip.io"
    
  3. Do a get on the Ingress.

    kubectl -n grafana get ingress grafana -o yaml
    

    It should look something like this.

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
    creationTimestamp: "2021-10-29T01:39:08Z"
    generation: 1
    name: grafana
    namespace: grafana
    resourceVersion: "136075"
    uid: 930c3017-bbc9-488c-a6f4-01fa82011d59
    spec:
    ingressClassName: contour
    rules:
    - host: grafana.18.189.222.216.nip.io
        http:
        paths:
        - backend:
            service:
                name: grafana
                port:
                number: 80
            path: /
            pathType: Prefix
    tls:
    - hosts:
        - grafana.18.189.222.216.nip.io
        secretName: grafana.18.189.222.216.nip.io
    status:
    loadBalancer:
        ingress:
        - hostname: a0a28d9b7f348457bafc05dd74e4e992-1884336594.us-east-2.elb.amazonaws.com
    
  4. Go to the following host path in a web browser to reach Grafana.

    INGRESS=$(kubectl -n grafana get ingress grafana -o jsonpath='{.spec.rules[0].host}')
    echo $INGRESS
    

    You will see the following where it says Not Secure.

    Browser URL bar

Next, you will make it so you can use HTTPS.

Issuers and certificates

OK, now back to cert-manager.

In order to issue certificates, you need to configure an Issuer. Some of the common types are SelfSigned, CA, and ACME. In this guide, you will be using an ACME (Automated Certificate Management Environment) Issuer. To ensure the client actually owns the domain that the client is requesting a certificate for, the ACME CA server has HTTP01 and DNS01 challenges. To learn more about these challenges, check out the cert-manager docs.

Create an ACME Issuer using Let’s Encrypt, a nonprofit Certificate Authority.

Let’s Encrypt recommends starting with the staging server to verify you have things right first and avoid rate limiting.

This Cluster Issuer uses a single challenge HTTP01 solver with Contour, your Ingress Controller.

Check out the docs to see more about Issuer Configuration. Here is an example of a resource definition for the Let’s Encrypt staging issuer.

  1. Replace the value of $CERT_EMAIL with your own email address.

    CERT_EMAIL=”user@example.com”
    
  2. Create a file called cm-clusterissuer-staging.yaml with the following.

    cat <<EOF > cm-clusterissuer-staging.yaml
    apiVersion: cert-manager.io/v1
    kind: ClusterIssuer
    metadata:
    name: letsencrypt-staging
    spec:
    acme:
    # Remember to update this if you use this manifest to obtain real certificates :)
    email: $CERT_EMAIL
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
        name: issuer-letsencrypt-staging
    solvers:
    - http01:
        ingress:
            class: contour
    EOF
    
  3. Create the issuer.

    kubectl apply -f cm-clusterissuer-staging.yaml
    
  4. Take a look and see the secret that is created.

    kubectl -n cert-manager get secret issuer-letsencrypt-staging -o yaml
    
  5. Once you have an Issuer configured properly, you can create a Certificate object. Certificates specify a secretName that cert-manager will either create or update with your new crypto identity. In your case, it’ll be the secret name in the Ingress you created, which is the value of $INGRESS.

  6. Check the secret name.

    kubectl -n grafana get ingress grafana -o jsonpath='{.spec.tls[0].secretName}'
    
  7. Create a Certificate object by creating a file called cm-certificate-staging.yaml with the following.

    cat << EOF > cm-certificate-staging.yaml
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
    name: $INGRESS
    namespace: grafana
    spec:
    secretName: $INGRESS
    dnsNames:
    - $INGRESS
    issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
    EOF
    
  8. Make sure that it filled in the names.

    cat cm-certificate-staging.yaml
    
  9. Apply the file.

    kubectl apply -f cm-certificate-staging.yaml
    

    You could instead add an annotation, and the certificate would be created for you.
    
    kubectl annotate ingress grafana -n grafana \
      "cert-manager.io/cluster-issuer=letsencrypt-staging"
    

  10. Wait for the certificate to be ready. Run Ctrl-C to terminate it when it’s ready.

    kubectl get certificate $INGRESS -w -n grafana
    
  11. You can see the secret in the default namespace.

    kubectl get secret | grep grafana
    
  12. Now, back in your web browser, change your URL to be https:// instead. You should see something similar to this where it issued a certificate, but it’s not valid. This is expected as you are still using the Let’s Encrypt staging issuer.

    Browser window Certificate window

  13. Create another ClusterIssuer, but this time with the production server.

    cat <<EOF > cm-clusterissuer-prod.yaml
    apiVersion: cert-manager.io/v1
    kind: ClusterIssuer
    metadata:
    name: letsencrypt-prod
    spec:
    acme:
    # Remember to update this if you use this manifest to obtain real certificates :)
    email: $CERT_EMAIL
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
        name: issuer-letsencrypt-prod
    solvers:
    - http01:
        ingress:
            class: contour
    EOF
    
  14. Apply the file.

    kubectl apply -f cm-clusterissuer-prod.yaml
    
  15. Switch staging to prod and apply to use the prod server.

    cat << EOF > cm-certificate-prod.yaml
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
    name: $INGRESS
    namespace: grafana
    spec:
    secretName: $INGRESS
    dnsNames:
    - $INGRESS
    issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
    EOF
    
  16. Apply the file.

    kubectl apply -f cm-certificate-prod.yaml
    
  17. Refresh your browser. Your HTTPS connection will be secure and have a valid certificate.

    Browser window Certificate window

Wrap-up

Congrats, you did it!

In a nutshell, you used cert-manager to make your web page secure by using HTTPS instead of just HTTP. Here’s what you just did. You…

  • Installed cert-manager
  • Installed Contour as your Ingress Controller
  • Created a web page (Grafana)
  • Created an Ingress
  • Created (ACME) Issuers and Certificates using Let’s Encrypt

Now if you have your own web pages running on Kubernetes, you can make them secure as well!

Want to play around some more? Check out the next section to actually get metric data into Grafana with Prometheus and also the Resources section to learn more.

Optional: Get metric data in Grafana

Set up Prometheus for metrics collection so you can actually see data in a dashboard in Grafana.

  1. List the packages available in your repository.

    tanzu package available list prometheus.community.tanzu.vmware.com
    
  2. Install Prometheus.

    tanzu package install prometheus \
    --package-name prometheus.community.tanzu.vmware.com \
    --version <VERSION>
    
  3. Make sure Prometheus is up and running.

    kubectl -n prometheus get pods
    
  4. Now go back to Grafana in the browser.

    echo https://$INGRESS
    
  5. If you haven’t already logged in, the default username and password are admin.

  6. Click on the word General in the top left corner

    Setting up Grafana

  7. Click on the second one that mentions Prometheus.

    Setting up Grafana

    You should now see a large dashboard with a bunch of charts and data to poke around in, such as the following.

    Setting up Grafana

    You can also create your own dashboards.

Resources