Kubeconform

Kubeconform is a Kubernetes manifests validation tool. Build it into your CI to validate your Kubernetes configuration using the schemas from the registry maintained by the kubernetes-json-schema project!

It is inspired by, contains code from and is designed to stay close to Kubeval, but with the following improvements:

• high performance: will validate & download manifests over multiple routines, caching downloaded files in memory
• configurable list of remote, or local schemas locations, enabling validating Kubernetes custom resources (CRDs)

A small overview of Kubernetes manifest validation

Kubernetes's API is described using the OpenAPI (formerly swagger) specification, in a file checked into the main Kubernetes repository.

Because of the state of the tooling to perform validation against OpenAPI schemas, projects usually convert the OpenAPI schemas to JSON schemas first. Kubeval relies on instrumenta/OpenApi2JsonSchema to convert Kubernetes' Swagger file and break it down into multiple JSON schemas, stored in github at instrumenta/kubernetes-json-schema and published on kubernetesjsonschema.dev.

Kubeconform relies on the same JSON schemas from kubernetesjsonschema.dev, and will download required schemas at runtime as required.

Limits of Kubeconform validation

Kubeconform, similarly to kubeval, only validates manifests using the OpenAPI specifications. In some cases, the Kubernetes controllers might perform additional validation - so that manifests passing kubeval validation would still error when being deployed. See for example these bugs against kubeval: #253 #256 #257 #259. The validation logic mentioned in these bug reports is not part of Kubernetes' OpenAPI spec, and therefore kubeconform/kubeval will not detect the configuration errors.

Usage

$ ./bin/kubeconform -h
Usage of ./bin/kubeconform:
  -ignore-missing-schemas
        skip files with missing schemas instead of failing
  -kubernetes-version string
        version of Kubernetes to validate against (default "1.18.0")
  -n int
        number of routines to run in parallel (default 4)
  -output string
        output format - text, json (default "text")
  -schema-location value
        override schemas location search path (can be specified multiple times)
  -skip string
        comma-separated list of kinds to ignore
  -strict
        disallow additional properties not in schema
  -summary
        print a summary at the end
  -verbose
        print results for all resources

Usage examples

• Validating a single, valid file

$ ./bin/kubeconform fixtures/valid.yaml
$ echo $?
0

• Validating a single invalid file, setting output to json, and printing a summary

$ ./bin/kubeconform -summary -output json fixtures/invalid.yaml
{
  "resources": [
    {
      "filename": "fixtures/invalid.yaml",
      "kind": "ReplicationController",
      "version": "v1",
      "status": "INVALID",
      "msg": "Additional property templates is not allowed - Invalid type. Expected: [integer,null], given: string"
    }
  ],
  "summary": {
    "valid": 0,
    "invalid": 1,
    "errors": 0,
    "skipped": 0
  }
}
$ echo $?
1

• Passing manifests via Stdin

cat fixtures/valid.yaml  | ./bin/kubeconform -summary -
Summary: 1 resource found parsing stdin - Valid: 1, Invalid: 0, Errors: 0 Skipped: 0

• Validating a folder, increasing the number of parallel workers

$ ./bin/kubeconform -summary -n 16 fixtures
fixtures/crd_schema.yaml - CustomResourceDefinition trainingjobs.sagemaker.aws.amazon.com failed validation: could not find schema for CustomResourceDefinition
fixtures/invalid.yaml - ReplicationController bob is invalid: Invalid type. Expected: [integer,null], given: string
[...]
Summary: 65 resources found in 34 files - Valid: 55, Invalid: 2, Errors: 8 Skipped: 0

Overriding schemas location - CRD support

When the -schema-location parameter is not used, kubeconform will default to downloading schemas from https://kubernetesjsonschema.dev. Kubeconform however supports passing one, or multiple, schemas locations - HTTP URLs, or local filesystem paths, in which case it will lookup for schema definitions in each of them, in order, stopping as soon as a matching file is found.

• If the -schema-location value does not end with '.json', Kubeconform will assume filenames / a file structure identical to that of kubernetesjsonschema.dev
• if the -schema-location value ends with '.json' - Kubeconform assumes the value is a Go templated string that indicates how to search for JSON schemas.

All 3 following command lines are equivalent:

$ ./bin/kubeconform fixtures/valid.yaml
$ ./bin/kubeconform -schema-location https://kubernetesjsonschema.dev fixtures/valid.yaml
$ ./bin/kubeconform -schema-location 'https://kubernetesjsonschema.dev/{{ .NormalizedVersion }}-standalone{{ .StrictSuffix }}/{{ .ResourceKind }}{{ .KindSuffix }}.json' fixtures/valid.yaml

To support validating CRDs, we need to convert OpenAPI files to JSON schema, storing the JSON schemas in a local folder - for example schemas. Then we specify this folder as an additional registry to lookup:

# If the resource Kind is not found in kubernetesjsonschema.dev, also lookup in the schemas/ folder for a matching file
$ ./bin/kubeconform -registry https://kubernetesjsonschema.dev -schema-location 'schemas/{{ .ResourceKind }}{{ .KindSuffix }}.json' fixtures/custom-resource.yaml

Openshift support

You can validate Openshift manifests using a custom schema location. Set the OpenShift version to validate against using -kubernetes-version.

bin/kubeconform -kubernetes-version 3.8.0  -schema-location 'https://raw.githubusercontent.com/garethr/openshift-json-schema/master/{{ .NormalizedVersion }}-standalone{{ .StrictSuffix }}/{{ .ResourceKind }}.json'  -summary fixtures/valid.yaml
Summary: 1 resource found in 1 file - Valid: 1, Invalid: 0, Errors: 0 Skipped: 0

Converting an OpenAPI file to a JSON Schema

Kubeconform uses JSON schemas to validate Kubernetes resources. For Custom Resource, the CustomResourceDefinition first needs to be converted to JSON Schema. A script is provided to convert these CustomResourceDefinitions to JSON schema. Here is an example how to use it:

$ ./cmd/openapi2jsonschema/main.py https://raw.githubusercontent.com/aws/amazon-sagemaker-operator-for-k8s/master/config/crd/bases/sagemaker.aws.amazon.com_trainingjobs.yaml > fixtures/registry/trainingjob-sagemaker-v1.json

Credits

• @garethr for the Kubeval and kubernetes-json-schema projects ❤️