---
weight: 1
---
# Creating a codegen pipeline

Pipelines are defined in YAML files usually named `cog.yaml`.

A typical pipeline will revolve around three main blocks:

* `inputs`: a list of schemas from which code will be generated
* `transformations`: modifications made by cog to the types described by the schemas and/or the builders it will generate. This block is optional.
* `output`: what should be generated, where, and in which language(s)

```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/grafana/cog/main/schemas/pipeline.json

debug: true

# `inputs` describes a list of schemas used by the pipeline to generate code.
inputs: []

# `transformations` is optional and holds transformations to apply to schemas
# and/or builders.
transformations:
  schemas: []

  builders: []

# `output` configures what will be generated by cog, where, and in which
# languages.
output:
  directory: './generated/%l'

  types: true
  builders: true
  converters: true
  api_reference: true

  languages: []
```

??? tip "Recommended: configuration validation and auto-complete"

    In order to help with configuring codegen pipelines, a [schema][schema.json] for
    `cog.yaml` files is provided. If your editor supports YAML schema validation,
    it is definitely recommended to set it up:

    === "Visual Studio Code"

        1.  Install [`vscode-yaml`][vscode-yaml] for YAML language support.
        2.  Add the schema under the `yaml.schemas` key in your user or
            workspace [`settings.json`][settings.json]:

            ``` json
            {
              "yaml.schemas": {
                "https://raw.githubusercontent.com/grafana/cog/main/schemas/pipeline.json": "cog.yaml"
              }
            }
            ```

    === "Other"

        1.  Ensure your editor of choice has support for YAML schema validation.
        2.  Add the following lines at the top of `cog.yaml`:

            ``` yaml
            # yaml-language-server: $schema=https://raw.githubusercontent.com/grafana/cog/main/schemas/pipeline.json
            ```

[schema.json]: https://raw.githubusercontent.com/grafana/cog/main/schemas/pipeline.json
[vscode-yaml]: https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml
[settings.json]: https://code.visualstudio.com/docs/getstarted/settings

## Example pipeline

The following pipeline takes the [schema exposed by cog to describe pipelines](https://raw.githubusercontent.com/grafana/cog/main/schemas/pipeline.json),
and generates types as well as builders from it in Go and Typescript:

```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/grafana/cog/main/schemas/pipeline.json

debug: true

# `inputs` describes a list of schemas used by the pipeline to generate code.
inputs:
  - jsonschema:
      url: https://raw.githubusercontent.com/grafana/cog/main/schemas/pipeline.json
      package: codegen

# `output` configures what will be generated by cog, where, and in which
# languages.
output:
  directory: './generated/%l'

  types: true
  builders: true

  languages:
    - go:
        package_root: 'github.com/owner/repo/generated/go'
    - typescript: {}
```

You can run it yourself with:

```shell
cog generate --config cog.yaml
```

## Built-in parameters

The following parameters are defined by `cog` and can be used within pipelines:

* `%__config_dir%`: refers to the directory containing the codegen pipeline.

## Inputs

Pipeline can rely on an arbitrary number of inputs, in any of schema languages
described below.

!!! tip

    A pipeline **can** mix inputs defined in different schema languages.

### CUE

```yaml
inputs:
  - cue:
      # Directory containing the CUE files to parse.
      # Required if `url` is empty.
      entrypoint: '%__config_dir%/schemas/cue_module'

      # URL to a cue file to parse.
      # Required if `entrypoint` is empty.
      url: 'https://raw.githubusercontent.com/grafana/grafana/v12.3.2/apps/dashboard/kinds/v2beta1/dashboard_spec.cue'

      # Path to the cue value to use as schema.
      subpath: 'manifestv1alpha1.schema'

      # Decorates the parsed cue Value with an envelope whose name is given.
      # This is useful for dataqueries for example, where the schema doesn't
      # define any suitable top-level object.
      forced_envelope: envelope_name

      # Package name to use for the input schema. If empty, it will be guessed
      # from the entrypoint.
      package: package_name

      # `cue_imports` allows importing additional libraries.
      # Format: [path]:[import]. Example: '../grafana/common-library:github.com/grafana/grafana/packages/grafana-schema/src/common
      cue_imports:
        - 'path/to/library:github.com/owner/library/path/to/cue/'

      # List of object names that will be allowed when parsing the input schema.
      # Note: if the list is empty/not specifiec, no filter is applied.
      allowed_objects:
        - ObjectName
        - OtherObjectName

      # List of paths to files containing schema transformations to apply to
      # this input.
      transforms:
        - '%__config_dir%/transformations/schemas/cue_module.yaml'
```

### JSON schema

```yaml
inputs:
  - jsonschema:
      # Path to a JSONSchema file.
      # Required if `url` is empty.
      path: '%__config_dir%/schemas/jsonschema.json'

      # URL to a JSONSchema file.
      # Required if `path` is empty.
      url: 'https://example.com/schemas/jsonschema.json'

      # Package name to use for the input schema. If empty, it will be guessed
      # from the entrypoint.
      package: package_name

      # List of object names that will be allowed when parsing the input schema.
      # Note: if the list is empty/not specifiec, no filter is applied.
      allowed_objects:
        - ObjectName
        - OtherObjectName

      # List of paths to files containing schema transformations to apply to
      # this input.
      transforms:
        - '%__config_dir%/transformations/schemas/jsonschema.yaml'
```

### OpenAPI

```yaml
inputs:
  - openapi:
      # Path to a OpenAPI file.
      # Required if `url` is empty.
      path: '%__config_dir%/schemas/openapi.json'

      # URL to a OpenAPI file.
      # Required if `path` is empty.
      url: 'https://example.com/schemas/openapi.json'

      # Package name to use for the input schema. If empty, it will be guessed
      # from the entrypoint.
      package: package_name

      # Disables the validation of the OpenAPI spec.
      # Default: false
      no_validate: false

      # List of object names that will be allowed when parsing the input schema.
      # Note: if the list is empty/not specifiec, no filter is applied.
      allowed_objects:
        - ObjectName
        - OtherObjectName

      # List of paths to files containing schema transformations to apply to
      # this input.
      transforms:
        - '%__config_dir%/transformations/schemas/openapi.yaml'
```

## Outputs

```yaml
output:
  # Path to a directory in which the generated code will be written.
  # Note: `%l` refers to the language name and means that the code for each
  # language will be written in its own sub-directory.
  directory: './generated/%l'

  # Generate types.
  types: true

  # Generate builders.
  builders: true

  # Generate converters.
  converters: true

  # Generate a markdown API reference.
  api_reference: true

  # List of languages to generate code for.
  languages: []
```

### Go

```yaml
output:
  # …

  languages:
    - go:
        # Root path for imports.
        # Required.
        package_root: 'github.com/owner/repo/generated/go'

        # GenerateJSONMarshaller controls the generation of `MarshalJSON()` and
        # `UnmarshalJSON()` methods on types.
        # Default: false
        generate_json_marshaller: true

        # Controls the generation of `UnmarshalJSONStrict()` methods on types.
        # Default: false
        generate_strict_unmarshaller: true

        # Controls the generation of `Equal()` methods on types.
        # Default: false
        generate_equal: true

        # Controls the generation of `Validate()` methods on types.
        # Default: false
        generate_validate: true

        # Instructs cog to emit `interface{}` instead of `any`.
        # Default: false
        any_as_interface: true

        # Disables runtime-related code generation.
        # Note: builders can NOT be generated with this flag turned on, as they
        # rely on the runtime to function.
        # Default: false
        skip_runtime: false

        # Disables the formatting of Go files done with `goimports` after code
        # generation.
        # Default: false
        skip_post_formatting: false

        # List of directories containing templates describing files to be added
        # to the generated output.
        extra_files_templates:
          - '%__config_dir%/templates/go/extra'

        # List of directories containing templates defining blocks used to
        # override parts of builders/types/....
        overrides_templates:
          - '%__config_dir%/templates/go/overrides'
```

### Java

```yaml
output:
  # …

  languages:
    - java:
        # Root path for imports.
        # Required.
        package_path: 'com.org.package'

        # Disables runtime-related code generation.
        # Note: builders can NOT be generated with this flag turned on, as they
        # rely on the runtime to function.
        # Default: false
        skip_runtime: false

        # List of directories containing templates describing files to be added
        # to the generated output.
        extra_files_templates:
          - '%__config_dir%/templates/java/extra'

        # List of directories containing templates defining blocks used to
        # override parts of builders/types/....
        overrides_templates:
          - '%__config_dir%/templates/java/overrides'
```

### JSON Schema

```yaml
output:
  # …

  languages:
    - jsonschema:
        # Controls whether the generated JSON should be pretty printed or not.
        compact: false
```

### OpenAPI

```yaml
output:
  # …

  languages:
    - openapi:
        # Controls whether the generated JSON should be pretty printed or not.
        compact: false
```

### PHP

```yaml
output:
  # …

  languages:
    - php:
        # Namespace root.
        # Required.
        namespace_root: 'Vendor\Package'

        # Controls the generation of `fromArray()` and `jsonSerialize()`
        # methods on types.
        # Default: false
        generate_json_marshaller: true

        # Allows to choose the name of the class that will be generated to
        # hold "builder factories".
        # By default, this class name is equal to the package name in which
        # factories are defined.
        # This option associates these package names with a class name.
        # Default: {}
        builder_factories_class_map: {}

        # List of directories containing templates describing files to be added
        # to the generated output.
        extra_files_templates:
          - '%__config_dir%/templates/php/extra'

        # List of directories containing templates defining blocks used to
        # override parts of builders/types/....
        overrides_templates:
          - '%__config_dir%/templates/php/overrides'
```

### Python

```yaml
output:
  # …

  languages:
    - python:
        # Prefix to add to every generated file path.
        # Default: ''
        path_prefix: ''

        # Controls the generation of `to_json()` and `from_json()` methods on
        # types.
        # Default: false
        generate_json_marshaller: true

        # Disables runtime-related code generation.
        # Note: builders can NOT be generated with this flag turned on, as they
        # rely on the runtime to function.
        # Default: false
        skip_runtime: false

        # List of directories containing templates describing files to be added
        # to the generated output.
        extra_files_templates:
          - '%__config_dir%/templates/python/extra'

        # List of directories containing templates defining blocks used to
        # override parts of builders/types/....
        overrides_templates:
          - '%__config_dir%/templates/python/overrides'
```

### Typescript

```yaml
output:
  # …

  languages:
    - typescript:
        # Prefix to add to every generated file path.
        # Default: 'src'
        path_prefix: 'src'

        # Disables runtime-related code generation.
        # Note: builders can NOT be generated with this flag turned on, as they
        # rely on the runtime to function.
        # Default: false
        skip_runtime: false

        # Disables the generation of `index.ts` files.
        # Default: false
        skip_index: false

        # Generates enums as a union of values instead of using an actual `enum`
        # declaration.
        # If enums_as_union_types is false, an enum will be generated as:
        #
        # ```ts
        # enum Direction {
        #   Up = "up",
        #   Down = "down",
        #   Left = "left",
        #   Right = "right",
        # }
        # ```
        #
        # If enums_as_union_types is true, the same enum will be generated as:
        #
        # ```ts
        # type Direction = "up" | "down" | "left" | "right";
        # ```
        #
        # Default: false
        enums_as_union_types: false

        # Associates package names to their import path.
        # Default: {}
        packages_import_map:
          'common': '@org/common'

        # List of directories containing templates describing files to be added
        # to the generated output.
        extra_files_templates:
          - '%__config_dir%/templates/typescript/extra'

        # List of directories containing templates defining blocks used to
        # override parts of builders/types/....
        overrides_templates:
          - '%__config_dir%/templates/typescript/overrides'
```

## Transformations

See ["Applying schema transformations"](./schema_transformations.md) and
["Applying builder transformations"](./builder_transformations.md).
