Tekton

Think of Tekton as the CI/CD engine that was born inside Kubernetes rather than bolted onto it. Jenkins grew up in the pre-container era, and GitHub Actions is a managed service locked inside GitHub. Tekton was designed from day one to speak Kubernetes natively, treating every pipeline step as a container running inside a pod, managed by the same scheduler and resource quotas that handle your production workloads.

Tekton is an open-source project originally developed at Google under the Knative project and later donated to the Continuous Delivery Foundation. It extends Kubernetes with a set of Custom Resource Definitions (CRDs) such as Task, TaskRun, Pipeline, PipelineRun, and Trigger. When you install Tekton on a cluster, the Tekton controller watches for these custom resources and orchestrates their execution. A Task defines a sequence of steps, and each step runs in its own container within a Kubernetes pod. A Pipeline chains multiple Tasks together, and a PipelineRun is the actual execution of that Pipeline. This is fundamentally different from Jenkins, where a Jenkinsfile describes stages that run on agents (either the controller node or separate worker nodes), or GitHub Actions, where workflow YAML files define jobs that execute on GitHub-hosted or self-hosted runners outside Kubernetes.

Under the hood, when you create a PipelineRun, the Tekton controller creates a TaskRun for each Task in the Pipeline. Each TaskRun becomes a Kubernetes pod, and each step within that Task becomes a container inside that pod. This means you get Kubernetes-native features for free: resource limits, node affinity, tolerations, service accounts, and RBAC all apply naturally. For example, when the payments-api team runs their build pipeline, Tekton schedules the build pod on a node with enough CPU and memory, using the same scheduling logic that places their production pods. Jenkins would need a separate plugin ecosystem to achieve even basic Kubernetes integration, and GitHub Actions requires self-hosted runners with custom configuration to run inside a cluster.

In production environments, Tekton shines when teams already run their applications on Kubernetes and want their CI/CD to live alongside their workloads. The order-processing-service team at a mid-size e-commerce company, for instance, can define their build, test, and deploy pipeline as Tekton resources stored in the same Git repository as their application code. Tekton Triggers can watch for GitHub webhooks and automatically create PipelineRuns, mimicking the event-driven model of GitHub Actions but running entirely within the cluster. The checkout-service team can reuse the same Task definitions for building container images, running integration tests, or deploying Helm charts across multiple pipelines.

One important distinction to keep in mind is the operational overhead trade-off. Jenkins and GitHub Actions abstract away the infrastructure: Jenkins gives you a web UI and plugin marketplace out of the box, and GitHub Actions requires zero infrastructure setup. Tekton requires you to manage the Tekton controller, maintain the cluster it runs on, handle storage for pipeline logs and artifacts, and build your own dashboard or use the optional Tekton Dashboard. This makes Tekton the stronger choice for Kubernetes-centric organizations that want full control but a heavier lift for small teams that just need a quick CI/CD solution without managing cluster infrastructure.

Think of a Task as a recipe card in a kitchen and a TaskRun as actually cooking that recipe. The recipe card lists the ingredients (inputs), the steps to follow (build, test, push), and the final dish (outputs). You can cook the same recipe multiple times with different ingredients, and each cooking session is a separate TaskRun that runs independently.

A Tekton Task is a Kubernetes Custom Resource that defines a sequence of steps to perform a specific unit of work. Each Task YAML manifest specifies an apiVersion (tekton.dev/v1), a kind (Task), metadata including the name, and a spec containing the list of steps. Each step specifies a container image and a script or command to run. Steps execute sequentially within a single pod, sharing the same network namespace and, optionally, the same volumes. For example, the inventory-sync team might define a Task called build-and-test that has three steps: one to install dependencies using a Node.js image, one to run unit tests, and one to run linting. Since all three steps run in the same pod, they share the workspace filesystem, so the dependencies installed in step one are available in step two without any special configuration.

Under the hood, when you create a TaskRun, the Tekton controller translates the Task definition into a Kubernetes pod specification. Each step becomes an init container or a regular container within that pod, and Tekton uses an internal entrypoint binary to enforce sequential execution. The TaskRun resource tracks the status of each step, recording start times, completion times, exit codes, and logs. If any step fails (returns a non-zero exit code), subsequent steps are skipped and the TaskRun is marked as failed. You can view the status of a TaskRun using kubectl get taskrun or the tkn CLI tool. The payments-api team, for instance, can run tkn taskrun logs build-and-test-run-7xk2m to stream the logs from a specific TaskRun and debug a failing test step.

In production workflows, Tasks are designed to be reusable building blocks. A Task like build-docker-image can accept parameters for the image name, tag, and Dockerfile path, allowing the user-auth-service, checkout-service, and order-processing-service teams to all use the same Task with different parameter values. The Tekton Hub (hub.tekton.dev) provides a catalog of community-maintained Tasks for common operations like building with Kaniko, pushing to registries, deploying with kubectl, and running security scans. Teams install these Tasks into their cluster and reference them in Pipelines, avoiding the need to write everything from scratch.

A common beginner mistake is confusing Tasks with Pipelines. A Task is a single unit of work executed in one pod, while a Pipeline orchestrates multiple Tasks across multiple pods. Another gotcha is that steps within a Task always run sequentially and cannot be parallelized. If you need parallel execution, you must use separate Tasks within a Pipeline and configure them to run concurrently. Also, remember that a TaskRun is immutable once created: you cannot update a running TaskRun, you must cancel it and create a new one. This immutability is a deliberate design choice that ensures reproducibility and auditability of every pipeline execution.

If a Task is a single workstation on an assembly line, a Pipeline is the entire factory floor layout that defines which workstations operate in which order, which ones can run in parallel, and how parts (data) move between stations. The PipelineRun is the actual production run where raw materials enter and a finished product comes out the other end.

A Tekton Pipeline is a Kubernetes Custom Resource that defines a collection of Tasks arranged in a specific execution order. Each Task within a Pipeline is referenced as a pipeline task (under the tasks array in the spec), and you can define dependencies between tasks using the runAfter field. Tasks without dependencies or runAfter constraints run in parallel by default. For example, the checkout-service team might define a Pipeline with four tasks: clone the source code, then run unit tests and security scanning in parallel (since neither depends on the other), and finally deploy to staging only after both tests and scanning succeed. This creates a DAG execution model where the Tekton controller determines the optimal execution order based on declared dependencies.

Internally, when you create a PipelineRun, the Tekton controller reads the Pipeline definition and creates a TaskRun for each pipeline task. Each TaskRun becomes its own Kubernetes pod, meaning tasks run in separate pods with separate filesystems and network namespaces. Data is shared between tasks through workspaces (backed by PersistentVolumeClaims) and results (small string values passed between tasks). The PipelineRun resource tracks the aggregate status: it is considered successful only if all TaskRuns succeed, and it fails if any required TaskRun fails. The user-auth-service team can inspect a PipelineRun to see which tasks completed, which are running, and which are pending, all through standard kubectl commands or the tkn CLI.

In production, Pipelines enable teams to model complex CI/CD workflows with clear separation of concerns. The order-processing-service team defines their deployment pipeline with stages for building the container image, running integration tests against a test database, scanning the image for vulnerabilities with Trivy, deploying to a staging environment, running smoke tests, and finally promoting to production with a manual approval gate. Each of these stages is a separate Task, and the Pipeline defines how they connect. Workspaces carry the source code and build artifacts through the pipeline, while results pass metadata like the built image digest from the build task to the deploy task.

A critical gotcha for beginners is understanding that tasks in a Pipeline run in separate pods, unlike steps within a Task that share a pod. This means you cannot simply write a file in one task and read it in the next unless they share a workspace backed by a PersistentVolumeClaim. Another common mistake is not setting runAfter correctly, which causes tasks to run in parallel when they should be sequential. Also, PipelineRuns support timeouts at both the pipeline level and individual task level, and beginners often forget to set these, leading to stuck pipelines that consume cluster resources indefinitely when a task hangs.

Imagine steps as workers sitting at the same desk in an office. Each worker (step) does their job one after the other, and because they share the same desk (pod filesystem), the first worker can leave documents (files) for the next worker to pick up. The desk drawers (workspaces) are shared storage, and the sticky notes (results) are small messages passed along.

In Tekton, a step is the smallest unit of execution. Each step is defined inside a Task spec and specifies a container image, a script or command to run, and optionally resource limits, environment variables, and volume mounts. Steps within a Task always execute sequentially in the order they are defined. When the Tekton controller creates a pod for a TaskRun, each step becomes a container within that pod. Tekton uses an internal entrypoint mechanism to ensure that step two does not start until step one has completed successfully. For example, in the payments-api build task, the first step clones the repository, the second step compiles the code, and the third step runs tests. Each step uses a different container image suited to its purpose: a git image for cloning, a golang image for compiling, and a testing image for running tests.

The primary mechanism for sharing data between steps is the shared filesystem within the pod. By default, Tekton mounts an emptyDir volume at a well-known path, and workspaces provide additional shared mount points. When the inventory-sync team defines a workspace called source, all steps in the Task can read from and write to $(workspaces.source.path). The first step clones the Git repository into this path, and subsequent steps can access the source code, build artifacts, and test results written there. Because all steps run in the same pod, this filesystem sharing is fast and does not require any network transfer. Additionally, Tekton provides a special /tekton/results directory where steps can write small string values (up to 4096 bytes) that become TaskRun results, which can then be referenced by other tasks in a Pipeline.

In production scenarios, teams use steps to decompose complex operations into discrete, debuggable units. The user-auth-service team might have a security scanning task with four steps: download the vulnerability database, scan dependencies, scan the container image, and generate a compliance report. Each step uses a specialized container image, but they all share the same workspace where the scan results accumulate. If the dependency scan step fails, the team can inspect the logs for that specific step without wading through the output of the entire task. The step-level granularity also makes it easy to add retry logic or timeout settings to individual operations.

A common beginner pitfall is trying to share data between steps using environment variables set in one step and read in another. Environment variables are scoped to a single container and do not persist across steps. Instead, you must write values to files in the shared workspace or use Tekton results. Another gotcha is that each step runs as a separate container with its own filesystem root, so a binary installed in step one (say, in /usr/local/bin) is not available in step two if it uses a different container image. The shared workspace is the only reliable mechanism for passing data. Finally, steps cannot run in parallel within a Task. If you need concurrent execution, you must split the work into separate Tasks within a Pipeline.

Installing Tekton is like setting up a new department in an office building. The core Pipelines component is the department itself, Triggers is the receptionist who handles incoming requests, the Dashboard is the monitoring screen in the lobby, and the tkn CLI is the intercom system that lets you communicate with the department from anywhere in the building.

Tekton Pipelines is the foundational component and is installed by applying a single YAML manifest from the official Tekton release bucket on Google Cloud Storage. The manifest creates the tekton-pipelines namespace, deploys the Tekton controller and webhook pods, registers the Custom Resource Definitions (Tasks, TaskRuns, Pipelines, PipelineRuns, etc.), and sets up the necessary RBAC roles and service accounts. The installation requires a Kubernetes cluster running version 1.25 or later, and the cluster must have a default StorageClass configured if you plan to use PersistentVolumeClaim-backed workspaces. For a production cluster running the payments-api and order-processing-service workloads, the Tekton controller typically consumes modest resources (around 100m CPU and 256Mi memory) but should be monitored as the number of concurrent PipelineRuns increases.

Beyond the core Pipelines component, teams typically install additional Tekton components based on their needs. Tekton Triggers enables event-driven pipeline execution by listening for incoming webhooks (such as GitHub push events) and automatically creating PipelineRuns. It introduces additional CRDs like EventListener, TriggerBinding, and TriggerTemplate. The Tekton Dashboard provides a web-based UI for viewing and managing Tasks, Pipelines, and their runs, which is helpful for teams that prefer a graphical interface over kubectl commands. The Tekton CLI (tkn) is a separate binary installed on developer machines that provides convenient commands for listing, starting, and monitoring pipeline resources without writing kubectl commands.

In production environments, the checkout-service team at a typical organization would install Tekton Pipelines and Triggers on their shared development cluster. The EventListener is configured to receive webhooks from the Git repository, so every push to a feature branch automatically triggers the CI pipeline. The user-auth-service team uses the Tekton Dashboard to monitor pipeline executions and quickly identify which commit broke the build. Platform engineers install the tkn CLI on CI bastion hosts and developer laptops for quick troubleshooting and ad-hoc pipeline management.

A common gotcha during installation is forgetting to verify that the tekton-pipelines namespace pods are in a Running state before attempting to create Tasks or Pipelines. The webhook pod must be ready to validate incoming resources, and attempting to apply a Task before the webhook is ready results in validation errors. Another mistake is installing Tekton on a cluster without a default StorageClass, which causes PVC-backed workspaces to hang in a Pending state. Always run kubectl get storageclass to verify before installation. For air-gapped or restricted environments, you can download the release YAML and container images ahead of time and host them in an internal registry, modifying the manifest to reference internal image paths.

Think of a workspace as a shared locker at a gym. The locker specification (workspace declaration) says you need a place to store your gear, but it does not specify whether the locker is a small one, a large one, or a temporary bag check. When you actually arrive (create a Run), you choose which locker to use (PVC, emptyDir, ConfigMap, or Secret). This abstraction lets the same workout routine (Task) work regardless of which locker you are assigned.

A Tekton Workspace is a volume declaration in a Task or Pipeline spec that provides a shared filesystem mount point for steps. In the Task definition, you declare the workspace with a name and an optional description, and then reference it in steps using the expression $(workspaces.<name>.path). When creating a TaskRun or PipelineRun, you bind the workspace to an actual volume source. The most common binding is a PersistentVolumeClaim (PVC), which provides durable storage that persists across steps and even across Tasks in a Pipeline. For example, the order-processing-service team declares a workspace called source in their build Task, and when they create a TaskRun, they bind it to a PVC that contains the cloned Git repository.

The workspace abstraction is critical because it decouples the Task definition from the storage implementation. The same git-clone Task can be used with a PVC when running in a production Pipeline (where data must persist across multiple Tasks), with an emptyDir when running a quick one-off test (where data can be discarded after the pod terminates), or with a ConfigMap when you need to inject static configuration files into the Task's filesystem. Tekton also supports volumeClaimTemplate, which automatically creates a temporary PVC for the duration of a PipelineRun and deletes it when the run completes. This is particularly useful for CI builds where you need persistent storage during the pipeline but do not want leftover PVCs cluttering the cluster.

In production workflows, the inventory-sync team uses workspaces to pass data through their deployment Pipeline. The first Task clones the repository into a PVC-backed workspace. The second Task reads the source code from that same workspace, builds a Docker image, and writes the image digest to a file in the workspace. The third Task reads the digest file and deploys the application. Without workspaces, each Task would run in its own pod with an isolated filesystem, and there would be no way to pass the built artifacts from the build Task to the deploy Task. The workspace backed by a PVC provides the shared filesystem bridge between these separate pods. Additionally, the user-auth-service team mounts a Kubernetes Secret as a read-only workspace to inject private SSH keys needed for cloning private Git repositories, keeping the credentials out of the Task definition.

A common beginner mistake is using emptyDir workspaces in a Pipeline with multiple Tasks. An emptyDir volume is scoped to a single pod, so data written by one Task in an emptyDir workspace is not visible to the next Task, which runs in a different pod. For multi-Task Pipelines, you must use a PVC or volumeClaimTemplate. Another gotcha is the access mode of the PVC: if you want multiple Tasks to access the same workspace concurrently (parallel tasks), the PVC must support ReadWriteMany (RWX) access mode, which not all storage providers support. Check your cluster's StorageClass capabilities before designing parallel workflows that share a workspace.

Parameters in Tekton work like function arguments in programming. The Task definition is the function signature that declares what inputs it expects, and the TaskRun is the function call that supplies the actual values. Just as a function deploy(service_name, environment) can be called with deploy('payments-api', 'staging') or deploy('checkout-service', 'production'), a parameterized Tekton Task can build and deploy any service without modifying the Task itself.

Tekton Parameters are declared in the params section of a Task or Pipeline spec. Each parameter has a name, a type (string or array), an optional description, and an optional default value. Within the Task's steps, parameters are referenced using the syntax $(params.<name>). When you create a TaskRun, you provide values for these parameters in the params section of the TaskRun spec. If a parameter has a default value and the TaskRun does not provide one, the default is used. If a parameter has no default and no value is provided, the TaskRun creation fails with a validation error. For example, the payments-api team defines a build-image Task with parameters for git-url, git-revision, and image-tag. Each time they create a TaskRun, they supply the specific branch and tag values for that build.

Parameters flow through the Tekton resource hierarchy in a structured way. A Pipeline declares its own parameters, which are separate from the parameters of the Tasks it contains. To pass a Pipeline-level parameter down to a Task, you explicitly map it in the Pipeline's task definition using the $(params.<name>) syntax. This explicit mapping prevents naming collisions and makes the data flow visible and auditable. The order-processing-service team defines a Pipeline with parameters for git-url, target-environment, and image-registry. Inside the Pipeline, the git-url parameter is mapped to the clone Task's url parameter, and the image-registry parameter is mapped to the build Task's destination parameter. This layered approach means changing the target environment is a single parameter change in the PipelineRun, not an edit to the Pipeline or Task definitions.

In production, parameters are the key mechanism for making Tasks and Pipelines reusable. The platform engineering team at a company running the user-auth-service, inventory-sync, and checkout-service creates a single generic-build-pipeline with parameters for the repository URL, branch, image registry, deployment namespace, and replica count. Each application team creates their own PipelineRuns with application-specific parameter values, all sharing the same Pipeline definition. This reduces duplication and ensures consistency: when the platform team adds a new security scanning step to the Pipeline, all application teams automatically get it on their next run. Parameters of type array are useful for passing multiple values, such as a list of test suites to run or a list of namespaces to deploy to.

A gotcha that trips up beginners is the difference between Tekton parameters and Kubernetes environment variables. Parameters are resolved at pipeline creation time by the Tekton controller and injected into the step scripts as literal string substitutions. They are not environment variables and cannot be accessed using $VAR syntax inside the container unless you explicitly set them as env vars in the step definition. Another common mistake is trying to use parameter values computed at runtime (like the output of a shell command) as input to another parameter. Parameters are static values set at TaskRun or PipelineRun creation time. For dynamic values computed during execution, use Tekton Results instead, which allow a step to write a value that a subsequent Task can consume.

If Tekton Pipelines is the engine room of a ship, the Dashboard is the bridge with its display screens showing engine status, speed, and navigation, while the tkn CLI is the walkie-talkie that lets the chief engineer bark commands from anywhere on the ship. Both provide visibility and control, but one is visual and the other is text-based. Most production teams use both: the Dashboard for at-a-glance monitoring and the CLI for scripted automation and quick troubleshooting.

The Tekton Dashboard is a Kubernetes-native web application that reads Tekton resources directly from the Kubernetes API server and presents them in a user-friendly interface. It shows a list of all Tasks, Pipelines, TaskRuns, and PipelineRuns across namespaces (if permissions allow), displays the DAG visualization of a Pipeline, shows real-time logs for running steps, and allows you to create new TaskRuns and PipelineRuns from the UI. The Dashboard is installed by applying its release YAML and is typically accessed through kubectl port-forward or an Ingress resource. For the payments-api team, the Dashboard provides a single pane of glass where a developer can see all recent pipeline runs, click into a failed run, identify which step failed, and read the error logs without ever touching kubectl.

The Tekton CLI (tkn) is a standalone binary distributed for Linux, macOS, and Windows. It communicates with the Kubernetes API server using the same kubeconfig as kubectl but provides Tekton-specific commands that are more ergonomic than raw kubectl commands. Instead of typing kubectl get pipelinerun -o yaml and parsing the YAML output to find the status, you type tkn pipelinerun list and get a clean table with names, statuses, and durations. The tkn pipeline start command is particularly useful: it creates a PipelineRun interactively, prompting you for parameter values and workspace bindings. The checkout-service team uses tkn pipeline start deploy-pipeline to kick off a deployment, and the CLI prompts them for the Git URL, branch, and target environment before creating the PipelineRun.

In production environments, the Dashboard and CLI serve complementary roles. The user-auth-service team has the Dashboard open on a shared monitor in their team area, showing the status of all pipeline runs. When a run fails, a developer uses tkn taskrun logs to stream the logs from the failed step directly to their terminal, which is faster than navigating the Dashboard UI for debugging. Platform engineers use tkn in shell scripts for automation: a nightly cron job runs tkn pipelinerun delete --keep 50 to clean up old PipelineRuns and prevent the cluster from accumulating thousands of completed run resources that consume etcd storage. The inventory-sync team uses tkn pipeline start in their deployment scripts, passing parameters programmatically rather than through the interactive prompt.

A gotcha worth noting is that the Tekton Dashboard does not provide authentication or authorization out of the box. It inherits the permissions of the service account it runs under, which by default has read access to Tekton resources in the tekton-pipelines namespace. For multi-tenant clusters where different teams should only see their own pipelines, you need to configure RBAC and potentially put the Dashboard behind an authentication proxy like OAuth2 Proxy or Dex. Another limitation is that the Dashboard does not persist logs after a pod is garbage collected by Kubernetes. For long-term log retention, teams integrate Tekton with external logging systems like Elasticsearch or Loki. The tkn CLI shares these limitations since it reads the same data from the Kubernetes API server.

Think of Tekton Triggers like a mail sorting facility at a post office. The EventListener is the mailbox that receives all incoming packages. The TriggerBinding is the sorting clerk who opens each package and reads the label to determine where it goes. The TriggerTemplate is the delivery instruction sheet that creates the actual delivery route based on the label information. Together, these three components transform raw webhook events into fully configured pipeline executions without any human intervention.

The EventListener is a Kubernetes Deployment and Service that Tekton creates when you apply an EventListener resource. It exposes an HTTP endpoint, typically on port 8080, that receives incoming webhooks from sources like GitHub, GitLab, Bitbucket, or any system capable of sending HTTP POST requests. When a push event arrives from GitHub for the payments-api repository, the EventListener receives the JSON payload and passes it through an optional chain of interceptors. Interceptors can validate the webhook signature using a shared secret, filter events based on criteria like branch name or event type, overlay additional fields onto the payload, or even call external services using a webhook interceptor. The built-in GitHub interceptor validates X-Hub-Signature headers, ensuring that only legitimate events from your configured GitHub webhook trigger pipeline runs, preventing unauthorized actors from flooding your cluster with rogue builds.

The TriggerBinding is a mapping document that extracts specific fields from the incoming event payload and assigns them to named parameters. Using JSONPath-style expressions, you can pull values like the repository URL from body.repository.clone_url, the commit SHA from body.head_commit.id, the branch name from body.ref, and the committer email from body.head_commit.committer.email. These extracted parameters are passed to the TriggerTemplate, which acts as a factory for Kubernetes resources. The TriggerTemplate defines a parameterized resourcetemplates section that typically contains a PipelineRun specification. When the EventListener processes an event, it merges the TriggerBinding parameters into the TriggerTemplate, rendering a complete PipelineRun manifest that Tekton then submits to the Kubernetes API server for execution.

In production, teams running microservices like order-processing-service and inventory-sync typically deploy a single EventListener per cluster or namespace, using CEL interceptors to route different repository events to different TriggerTemplates. A CEL filter expression like body.repository.name == 'order-processing-service' && body.ref == 'refs/heads/main' ensures that only main branch pushes to the order processing repository trigger the production deployment pipeline. For the inventory-sync service, a separate trigger within the same EventListener routes its events to a different pipeline that includes database migration steps. This fan-out architecture avoids deploying dozens of EventListeners while maintaining clean separation of pipeline logic per service.

A critical production gotcha is securing the EventListener endpoint. Without webhook secret validation, anyone who discovers the endpoint URL can trigger pipeline runs by sending crafted payloads. Always configure the GitHub interceptor with a secretRef pointing to a Kubernetes Secret containing the webhook secret token. Additionally, EventListeners consume cluster resources proportional to incoming event volume. For high-traffic repositories with frequent pushes, configure horizontal pod autoscaling on the EventListener Deployment and set resource requests and limits appropriately. Monitor the EventListener pods for memory pressure, as large payloads from monorepo webhooks can cause OOM kills if memory limits are set too low. Use the tekton-triggers-controller logs and the EventListener sink logs to debug event processing failures, and always test your JSONPath expressions against real webhook payloads before deploying to production.

Think of Tekton Results like sticky notes that workers leave on a shared bulletin board in a factory assembly line. Each worker completes their station, writes a note with critical information like a part number or quality measurement, and pins it to the board. The next worker downstream reads those notes before starting their own job. In Tekton, Results are the mechanism that lets one Task produce output data that subsequent Tasks can consume as input, enabling dynamic data flow across an otherwise statically defined Pipeline.

At the Task level, you declare Results in the spec.results section with a name and an optional description. During execution, the Task step writes a value to a file at the path /tekton/results/<result-name>. The Tekton controller reads this file after the step container completes and stores the value as part of the TaskRun status. The value must be a string and, importantly, is subject to a size limit. In Tekton Pipelines v0.44 and later, individual results can be up to 4096 bytes by default, though this limit is configurable via the feature-flags ConfigMap. For larger data, you should write to a workspace volume and pass a file path as the result instead of the data itself. Results are terminated at the exact byte boundary of the file content, so avoid trailing newlines unless they are intentional, as they become part of the stored value.

Within a Pipeline, you reference a Task result using the variable substitution syntax $(tasks.<task-name>.results.<result-name>). This creates an implicit ordering dependency: the consuming Task will not start until the producing Task has completed successfully. For example, in a payments-api pipeline, a build-image Task can emit the full image digest as a result, and the subsequent deploy Task consumes that digest to ensure it deploys the exact image that was just built, not a mutable tag that could have been overwritten. This pattern eliminates the race condition where a deploy step pulls a latest tag that points to a different build than the one just tested. The Pipeline can also emit its own results by mapping Task results to Pipeline-level results in spec.results, making them accessible to external systems querying the PipelineRun status.

In production microservice workflows, Results become the glue connecting stages of complex pipelines. Consider a pipeline for the order-processing-service that first runs a generate-version Task to compute the semantic version from git tags, producing a result called version. The next Task, build-and-push, consumes that version to tag the container image and emits a result called image-digest containing the SHA256 digest. A parallel security-scan Task consumes the image-digest to scan the exact image for vulnerabilities and emits a scan-status result of either pass or fail. The deploy Task uses a when expression to check if scan-status equals pass before proceeding with the deployment, consuming the image-digest result to update the Kubernetes Deployment manifest with the immutable digest reference. This chain of Results creates a verifiable audit trail where every artifact is traceable from source commit to running container.

A common gotcha is attempting to use Results for large data transfers like test reports, coverage files, or build logs. Results are designed for small metadata values such as image digests, version strings, commit SHAs, or status flags. Writing megabytes of data to /tekton/results will either be truncated or cause the TaskRun to fail with a result size exceeded error. For large artifacts, use a shared workspace backed by a PersistentVolumeClaim where one Task writes files and the next Task reads them from the same mounted volume. Another pitfall is forgetting that Results create implicit dependencies. If Task B references a result from Task A, Task B cannot run in parallel with Task A even if you intended them to be parallel. Always review the Pipeline execution graph after adding result references to verify that the parallelism and ordering match your expectations. Use tkn pipelinerun describe to inspect result values after execution for debugging.

Think of When Expressions like a bouncer at a club entrance. Before each Task is allowed to execute, the bouncer checks the guest list. If the Task passes the check, it enters and runs. If it fails the check, it is politely turned away and marked as skipped rather than failed. This skip behavior is important because it means the overall Pipeline can still succeed even when some Tasks are conditionally skipped, unlike a failed Task which would cause the Pipeline to fail.

When Expressions are defined at the Task level within a Pipeline using the when field. Each when expression consists of three parts: an input value that can be a static string or a variable reference, an operator which is either In or NotIn, and a values array containing the allowed or disallowed strings to compare against. The input is evaluated against the values list using the specified operator. If all when expressions on a Task evaluate to true, the Task executes. If any single when expression evaluates to false, the entire Task is skipped. The input field commonly references Pipeline parameters using $(params.paramname), Task results using $(tasks.taskname.results.resultname), or context variables like $(context.pipeline.name). This makes when expressions dynamic and responsive to runtime values rather than being hardcoded at definition time.

In production pipelines, when expressions enable sophisticated branching patterns without requiring separate Pipeline definitions for each workflow variation. Consider a CI/CD pipeline for the checkout-service where the team wants to run integration tests only on pull request branches but skip them on feature branches to save cluster resources. A when expression with input $(params.branch) operator In values ["refs/heads/main", "refs/heads/release/*"] on the integration-test Task ensures that only main and release branches get the full test suite. For the deploy-to-staging Task, a when expression checks that the preceding security-scan Task produced a result of pass using input $(tasks.security-scan.results.status) operator In values ["pass"]. If the scan found critical vulnerabilities and reported fail, the deployment Task is skipped entirely, preventing vulnerable code from reaching any environment.

The behavior of downstream Tasks when an upstream Task is skipped deserves special attention. In Tekton v0.44 and later, you can configure the scope-when-expressions-to-task feature flag to control whether a skipped Task causes its downstream dependents to also be skipped. With this flag set to true, only the Task with the failing when expression is skipped, and downstream Tasks that depend on it via runAfter but do not reference its results will still execute. Without this flag, the default behavior cascades the skip to all downstream Tasks, effectively creating a short-circuit in the Pipeline. Understanding this distinction is crucial for designing pipelines where some branches should continue even when a conditional Task is skipped. For instance, in an order-processing-service pipeline, you might want the notification Task to run regardless of whether the optional performance-test Task was skipped.

A common mistake is trying to use when expressions for complex boolean logic with AND and OR combinations. When expressions only support simple equality checks with In and NotIn operators. If you need to evaluate whether a branch is main AND the test result is pass, you add two separate when expressions to the same Task, because multiple when expressions are evaluated with AND logic. For OR logic, you must restructure your Pipeline or use a preceding Task that evaluates the complex condition and writes a single result that the when expression can check. Another gotcha is that skipped Tasks do not produce results, so any downstream Task referencing results from a potentially skipped Task will fail with a missing result error. Always design your Pipeline DAG so that Tasks consuming results only depend on Tasks that are guaranteed to execute. Use the tkn pipelinerun describe command to inspect which Tasks were skipped and why, which is invaluable for debugging conditional execution flows in production.

Think of managing secrets in Tekton like handling keys in a large hotel. The front desk has a master key cabinet that guests never access directly. Instead, the concierge hands each guest only the specific room key they need, and the key is collected when they check out. In Tekton, Kubernetes Secrets are the key cabinet, service accounts are the concierge, and Task steps are the guests who receive only the credentials they need for the duration of their execution.

The most fundamental approach is creating Kubernetes Secret resources and referencing them in Task step definitions. You can mount secrets as environment variables using the envFrom or env with valueFrom.secretKeyRef syntax, or as files using volume mounts with a secret volume type. For the payments-api pipeline, a database migration step needs the DATABASE_URL connection string. Rather than hardcoding it in the Pipeline definition, you create a Secret containing the connection string and reference it in the step's env block. The step container sees the credential as a normal environment variable, and the Secret value never appears in Pipeline YAML, TaskRun logs, or Tekton dashboard displays. For credentials that must be files, such as GCP service account JSON keys or TLS certificates, volume-mounting the Secret to a specific path like /etc/secrets/sa-key.json gives the step direct file access while keeping the credential managed through Kubernetes RBAC.

Tekton provides a specialized mechanism for Git and Docker registry authentication through annotated Secrets linked to service accounts. By creating a Secret of type kubernetes.io/basic-auth with a tekton.dev/git-0 annotation pointing to the Git server URL, and attaching it to the service account used by the TaskRun, Tekton automatically configures Git credential helpers inside every step container. The same pattern works for Docker registries using kubernetes.io/dockerconfigjson secrets with tekton.dev/docker-0 annotations. When the user-auth-service pipeline clones a private repository and pushes images to a private registry, the service account carries both the Git credentials for cloning and the Docker credentials for pushing, without any explicit Secret references in the Task definition. This declarative approach centralizes credential management at the service account level and decouples Task definitions from specific credential implementations.

For enterprise environments, integrating external secret managers provides dynamic credential rotation and fine-grained audit trails that Kubernetes Secrets alone cannot offer. HashiCorp Vault integration typically uses the Vault Agent sidecar injector, which adds init containers and sidecar containers to TaskRun pods that fetch secrets from Vault and write them to a shared in-memory volume. The External Secrets Operator is another popular approach that synchronizes secrets from AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager into Kubernetes Secret resources, which Tekton Tasks then consume through standard volume mounts or environment variables. For the inventory-sync service that connects to third-party supplier APIs with rotating API keys, the External Secrets Operator ensures that the Kubernetes Secret always contains the current API key, and Tekton Tasks pick up the fresh credentials on every run without pipeline changes.

Security best practices demand treating Tekton secret management as a defense-in-depth problem. Never echo secret values in step scripts, as TaskRun logs are accessible through kubectl and the Tekton dashboard. Use Kubernetes RBAC to restrict which service accounts can access which Secrets, applying the principle of least privilege so that the checkout-service pipeline cannot read Secrets belonging to the payments-api pipeline. Enable encryption at rest for etcd to protect Secret values stored in the cluster. Rotate credentials regularly and use short-lived tokens where possible, such as OIDC tokens for cloud provider authentication instead of long-lived service account keys. Tekton Chains can sign TaskRun and PipelineRun attestations, providing a verifiable record that specific credentials were used during specific pipeline executions, supporting compliance requirements for SOC 2 and PCI DSS environments.

Think of building container images in Tekton like assembling a product in a clean-room factory. Traditional Docker builds require access to the Docker daemon, which is like needing the factory's master control panel to assemble a single product. Kaniko and Buildah are self-contained assembly stations that can build the product independently without needing the master control panel, making them safe to operate in shared environments where giving every worker access to the master controls would be a security risk.

Kaniko is the most popular image builder in Tekton pipelines because it runs entirely in userspace without requiring a Docker daemon or privileged container mode. The Kaniko executor reads a Dockerfile from the workspace, executes each instruction in a sandboxed environment, and pushes the resulting image layers directly to a container registry. In a Tekton Task, you create a step using the gcr.io/kaniko-project/executor image, pass the Dockerfile location and build context via arguments, and specify the destination registry and tag with the --destination flag. Kaniko supports multi-stage builds, build arguments, layer caching through a remote cache repository, and digest file output that integrates perfectly with Tekton Results. For the payments-api service, Kaniko reads the Dockerfile from the cloned source workspace, builds the Go binary in a builder stage, copies it into a distroless runtime image, and pushes the result to registry.acme.io/payments-api with both a git-sha tag and a semver tag.

Buildah offers an alternative approach that provides more flexibility and closer compatibility with Docker build behavior. Buildah supports Dockerfile-based builds through the buildah bud command, but also supports building images programmatically using buildah from, buildah copy, buildah run, and buildah commit commands. This programmatic approach is valuable when you need to construct images based on dynamic conditions that are difficult to express in a Dockerfile. Buildah can run without root privileges using the --storage-driver=vfs flag, though it typically needs some elevated capabilities. In Tekton, Buildah steps use the quay.io/buildah/buildah image and mount the container storage as an emptyDir volume. For the user-auth-service, a Buildah-based Task can conditionally include debug tools in development builds while producing minimal images for production, all within the same Task definition using script logic.

Registry authentication for both Kaniko and Buildah is handled through Kubernetes Secrets. Kaniko reads Docker credentials from a config.json file, which you provide either through Tekton's automatic credential injection via annotated Secrets on the service account, or by explicitly mounting a dockerconfigjson Secret to /kaniko/.docker/config.json. Buildah uses the --authfile flag pointing to a mounted credentials file. For multi-registry workflows where the checkout-service pipeline pulls base images from Docker Hub, caches layers in a GCR cache repository, and pushes the final image to a private ECR registry, you create separate Secrets for each registry and mount them into the builder step. Kaniko supports multiple --destination flags to push the same image to multiple registries in a single build, which is useful for disaster recovery setups where images must exist in two geographically separated registries.

Performance optimization is critical for production Tekton image builds. Kaniko's --cache=true flag enables layer caching using a remote registry as the cache backend, dramatically reducing build times for images where early layers like dependency installation rarely change. The --cache-repo flag specifies where cached layers are stored. For the order-processing-service with a large node_modules layer, enabling caching reduces build times from eight minutes to under two minutes when only application code changes. Buildah benefits from mounting a persistent cache volume for the package manager cache directory. Both tools support --snapshot-mode=redo in Kaniko and equivalent optimizations in Buildah to skip unchanged layers more efficiently. Always output the image digest to /tekton/results so downstream Tasks can reference the exact immutable image rather than a mutable tag that could be overwritten by a concurrent build.

Think of the Tekton Catalog like an app store for CI/CD building blocks. Instead of every team writing their own git-clone Task, image-build Task, and deployment Task from scratch, the Tekton Hub provides pre-built, community-tested components that you can install with a single command. Just as you would not write your own web browser when you can download one, you should not write your own git-clone Task when a battle-tested version already exists in the catalog.

The Tekton Catalog lives in the tektoncd/catalog GitHub repository and is searchable through hub.tekton.dev, the official Tekton Hub web interface. The catalog organizes Tasks and Pipelines by category, including git operations, image building, testing frameworks, deployment tools, notification services, and cloud provider integrations. Each catalog entry includes a README with usage instructions, parameter documentation, workspace requirements, and example PipelineRun manifests. Tasks are versioned following semantic versioning, allowing you to pin specific versions in your pipelines for reproducibility. The git-clone Task, for example, has gone through multiple major versions, each improving performance, adding features like sparse checkout and submodule support, and fixing edge cases discovered by the community.

Installing a catalog Task into your cluster can be done through several methods. The simplest approach uses kubectl apply with the raw GitHub URL pointing to the specific Task version YAML. The tkn CLI provides a more ergonomic experience with tkn hub install task git-clone which automatically resolves the latest version and applies it to your cluster. For production environments, the recommended approach is to use Tekton Bundles, which package Task definitions as OCI artifacts stored in your container registry. This ensures that your pipeline references are immutable and versioned, preventing a scenario where someone updates the catalog Task in-place and breaks running pipelines. When the checkout-service pipeline references the git-clone Task, using a bundle reference guarantees that every execution uses the exact same Task definition regardless of what happens in the upstream catalog.

Customizing catalog Tasks is a common requirement since generic Tasks rarely match every team's exact needs. Rather than forking and modifying catalog Tasks directly, the recommended pattern is to wrap them in a Pipeline that provides organization-specific defaults and add custom pre-processing or post-processing steps as separate Tasks. For instance, the generic git-clone Task works for any repository, but your organization might need to configure a corporate proxy, use an internal certificate authority, or apply specific Git LFS settings. Instead of modifying git-clone, create a Pipeline that runs a configure-git-environment Task before git-clone, setting up the necessary environment. If the catalog Task truly needs modification, vendor it into your organization's internal Task registry by copying it, making changes, and publishing it as a Tekton Bundle to your private registry. This gives you full control while maintaining a clear lineage back to the original catalog Task.

In production, teams managing microservices like payments-api, order-processing-service, and inventory-sync benefit enormously from standardized catalog Tasks. The platform engineering team maintains an internal catalog of blessed Tasks that have been security-reviewed, performance-tested, and configured with organization-specific defaults. Each service team builds their Pipeline by composing these standard Tasks rather than writing custom implementations. This standardization reduces maintenance burden, ensures consistent security practices across all pipelines, and accelerates onboarding of new services. When a vulnerability is discovered in a catalog Task, the platform team updates it once in the internal registry, and all pipelines pick up the fix on their next run. Monitor catalog Task usage across your cluster with Tekton Dashboard and set up alerts for deprecated Task versions to maintain hygiene across dozens of pipelines.

Think of a Tekton CI/CD pipeline for microservices like an automotive assembly line. Each station on the line performs a specialized task: welding, painting, quality inspection, and final assembly. Parts move between stations on a conveyor belt. In Tekton, each station is a Task, the conveyor belt is a workspace, quality gates are when expressions, and the factory floor manager who decides when to start the line is a Trigger. The result is a fully automated, repeatable process that transforms source code into a running, tested, production-ready service.

The pipeline architecture for a microservice like payments-api typically consists of six to eight sequential and parallel Tasks. The first Task, git-clone from the Tekton Catalog, checks out the source code into a shared workspace. The next stage fans out into parallel Tasks: a lint-and-format Task checks code style, a unit-test Task runs the test suite, and a dependency-audit Task scans for vulnerable dependencies. These three can run simultaneously because they all read from the same workspace without conflicting writes. After all three parallel Tasks succeed, the pipeline converges on a build-and-push Task that uses Kaniko to create a container image and push it to the registry, emitting the image digest as a Tekton Result. A security-scan Task then runs Trivy or Grype against the built image using the digest result, and emits a scan-status result. Finally, a deploy Task uses kubectl or Helm to update the Kubernetes Deployment with the new image digest, but only if the security scan passed, enforced by a when expression.

Workspace design is critical for microservice pipelines. A single PersistentVolumeClaim workspace shared across all Tasks provides the source code, build artifacts, and test reports. Using a volumeClaimTemplate in the PipelineRun automatically provisions a fresh PVC for each run, ensuring isolation between concurrent pipeline executions for different services. The access mode must be ReadWriteOnce for most cloud providers, which means all Tasks sharing the workspace must run on the same node. If your cluster uses multiple availability zones, configure a StorageClass with volumeBindingMode: WaitForFirstConsumer to ensure the PVC is created in the zone where the first Task pod is scheduled. For the order-processing-service, which has a large test data directory, allocating a 5Gi PVC prevents out-of-space errors during parallel test execution.

In a microservices architecture with services like payments-api, user-auth-service, order-processing-service, inventory-sync, and checkout-service, you need a strategy for managing multiple pipelines efficiently. The recommended approach is creating a parameterized Pipeline template that accepts the repository URL, image name, deployment name, and namespace as parameters. Each service uses the same Pipeline definition with different PipelineRun parameters, avoiding pipeline duplication. Tekton Triggers handles the automation: a single EventListener receives webhooks from all repositories, CEL interceptors route events to the correct TriggerTemplate based on the repository name, and each TriggerTemplate creates a PipelineRun with service-specific parameters. This architecture means adding a new microservice to the CI/CD system requires only a new TriggerTemplate and TriggerBinding, not a new Pipeline.

Production hardening involves several critical considerations. Set resource requests and limits on every Task step to prevent build pods from consuming excessive cluster resources and starving other workloads. Configure timeouts at both the Task and Pipeline levels to prevent hung builds from occupying cluster resources indefinitely. Use the finally block in the Pipeline to define cleanup Tasks that run regardless of pipeline success or failure, such as sending Slack notifications, updating deployment status in a dashboard, or cleaning up temporary cloud resources. Implement Tekton Chains for supply chain security, which automatically signs TaskRun and PipelineRun attestations and generates SLSA provenance documents. For the checkout-service handling financial transactions, this provenance chain provides auditable proof that every deployed image was built from a specific commit, passed all tests, cleared the security scan, and was deployed by an authorized pipeline execution.

Think of debugging Tekton pipelines like troubleshooting a car that will not start. You do not immediately tear apart the engine. Instead, you follow a systematic diagnostic checklist: check fuel, check battery, check starter motor, check spark plugs. Each check narrows down the problem. In Tekton, you follow a similar layered approach: start with the PipelineRun status, drill into the specific failed TaskRun, examine the step container logs, and inspect the underlying pod events. This top-down methodology prevents wasting time on irrelevant components.

The first diagnostic step is examining the PipelineRun or TaskRun status using kubectl describe or tkn pipelinerun describe. The status section contains a conditions array with the overall run status, a childReferences or taskRuns map showing the status of each individual Task, and timestamps for when each Task started and completed. A PipelineRun can fail for several reasons: a Task step exited with a non-zero exit code, a when expression evaluated to false causing unexpected skips, a timeout was exceeded, or a workspace binding failed. The tkn pipelinerun describe command provides a human-readable summary showing which Tasks succeeded with a checkmark, which failed with an X, which were skipped, and which timed out. For the payments-api pipeline, if the security-scan Task failed, this summary immediately tells you where to focus your investigation.

Drilling into the failed TaskRun reveals step-level details. Each Task can contain multiple steps, and Tekton tracks the exit code and termination reason for every step container. The tkn taskrun logs command streams the logs from all step containers in execution order, or you can target a specific step with the --step flag. For long-running Tasks in the order-processing-service pipeline, filtering logs to the failing step avoids scrolling through megabytes of irrelevant output. If the step container was OOMKilled, the termination reason in the TaskRun status will indicate this, pointing to a resource limits issue rather than a code bug. If the step exited with a specific error code, the container logs usually contain the stack trace or error message that explains the failure.

Pod-level issues represent a separate class of failures that manifest before any step container even starts. Common pod-level problems include ImagePullBackOff when the step container image cannot be pulled from the registry, pending pods due to insufficient cluster resources or node affinity rules that cannot be satisfied, PVC binding failures when the workspace volume cannot be provisioned, and service account permission errors when the TaskRun SA lacks RBAC permissions to create or access resources. Use kubectl describe pod on the TaskRun pod to inspect the events section, which contains detailed messages about scheduling decisions, image pulls, volume mounts, and container starts. For the user-auth-service pipeline, an ImagePullBackOff on the custom test runner image indicates either incorrect image tags, missing registry credentials, or a registry outage, and the pod events will specify which.

Advanced debugging techniques include using the Tekton Dashboard for visual pipeline execution tracing, which shows the DAG of Tasks with color-coded statuses and clickable log viewers. For intermittent failures, examine the TaskRun's retries field if retry policies are configured, and compare timestamps between the failing run and surrounding successful runs to identify environmental changes. Enable verbose logging in the Tekton controller by modifying the config-logging ConfigMap in the tekton-pipelines namespace, which provides detailed information about Pipeline validation, Task scheduling, and result propagation. For workspace-related failures, exec into a debug pod using the same PVC to inspect file permissions, disk space, and directory structure. Always check whether the failure is deterministic by re-running the PipelineRun with the same parameters, as transient failures from network timeouts, registry rate limits, or resource pressure require different remediation than code-level bugs.

Tekton Chains addresses one of the most critical challenges in modern software delivery: proving that a container image or artifact was actually built by your CI/CD system and has not been tampered with between build and deployment. In a world where supply chain attacks like SolarWinds and Codecov have compromised thousands of organizations, simply building an image is no longer sufficient. You must cryptographically prove who built it, what source code was used, what build steps were executed, and that the resulting artifact is identical to what was produced. Tekton Chains automates this entire provenance chain without requiring developers to modify their pipelines.

Tekton Chains operates as a Kubernetes controller that watches for completed TaskRuns and PipelineRuns. When the payments-api pipeline finishes building a container image, Chains automatically captures the build metadata including the git commit SHA, the Dockerfile used, the base image digest, and all input parameters. It then generates an in-toto attestation document conforming to the SLSA (Supply-chain Levels for Software Artifacts) framework. This attestation is cryptographically signed using either a locally stored key pair, a KMS-managed key from AWS KMS, GCP KMS, or HashiCorp Vault, or Sigstore's Fulcio for keyless signing where the identity is bound to an OIDC token from the Kubernetes service account. The signed attestation is then stored alongside the artifact in the OCI registry, attached as a cosign signature or uploaded to a Rekor transparency log for public verifiability.

In a production environment at enterprise scale, implementing Chains for the order-processing-service and inventory-sync pipelines requires careful architecture decisions. The signing key management strategy is paramount. Organizations handling financial transactions through their checkout-service typically use hardware-backed KMS keys with strict IAM policies, ensuring that only the Tekton Chains controller service account can access the signing key. The transparency log integration means every signed artifact is recorded in an immutable append-only ledger, making it impossible to retroactively modify or deny that a particular build occurred. For the user-auth-service, this creates a complete audit trail from git commit to running container that satisfies SOC 2, FedRAMP, and PCI-DSS compliance requirements without manual evidence collection.

The verification side of the equation is equally important. When ArgoCD or a Kubernetes admission controller like Kyverno or OPA Gatekeeper attempts to deploy the payments-api image, it first verifies the cosign signature and validates the SLSA attestation against organizational policies. These policies can enforce requirements such as the image was built from the main branch of an approved repository, the build pipeline included a security scanning step that passed, the source code had at least two approved reviewers, and the build occurred on a trusted build cluster. If any of these conditions fail verification, the deployment is blocked before the image ever reaches the production cluster. This creates a cryptographic trust chain from developer commit through CI/CD build to production deployment.

The operational complexity of Tekton Chains centers around key rotation, attestation format evolution, and multi-cluster deployment. Key rotation requires updating the signing key across all clusters while maintaining the ability to verify artifacts signed with the previous key during the rotation window. Organizations running the inventory-sync and checkout-service across multiple regions deploy Chains in each cluster with region-specific KMS keys, using a federated trust model where each regional key is registered as a trusted signer in the verification policies of all clusters. The SLSA framework defines four levels of assurance, and reaching SLSA Level 3 requires that the build platform itself is hardened, the build definition comes from a version-controlled source, and the provenance is non-falsifiable. Tekton Chains provides the provenance generation and signing infrastructure, but achieving SLSA Level 3 also requires securing the Tekton controller, isolating build namespaces, and ensuring that pipeline definitions cannot be modified at runtime.

Enterprise Tekton scaling is fundamentally a Kubernetes multi-tenancy problem compounded by the unique characteristics of CI/CD workloads: bursty resource consumption, long-running processes that compete for cluster capacity, and the need for strong isolation between teams whose pipelines may process sensitive code and credentials. When the payments-api team triggers 50 parallel pipeline runs during a release sprint while the user-auth-service team is running integration tests, the platform must ensure fair resource distribution, prevent noisy-neighbor problems, and maintain security boundaries between tenant workloads without requiring manual intervention from platform engineers.

The namespace isolation model forms the foundation of enterprise Tekton multi-tenancy. Each team or business unit receives a dedicated namespace such as ci-payments-team, ci-checkout-team, and ci-platform-team where their PipelineRuns execute. Kubernetes ResourceQuotas are applied to each namespace to cap the total CPU, memory, and ephemeral storage that a team's pipeline runs can consume simultaneously. LimitRanges set default and maximum resource requests for individual TaskRun pods, preventing a single poorly configured task from consuming an entire node. The Tekton controller itself runs in a separate administrative namespace with elevated permissions, while tenant namespaces contain only the PipelineRuns, TaskRuns, and their associated pods. This separation means that even if a malicious or misconfigured pipeline in the inventory-sync namespace attempts to escalate privileges, namespace-scoped RBAC policies and pod security standards prevent it from accessing resources in other tenant namespaces or the Tekton controller namespace.

Resource management at enterprise scale requires moving beyond simple quotas to intelligent scheduling and priority-based queuing. Tekton's concurrency controls allow setting maximum concurrent TaskRuns per namespace, but production environments need more sophisticated approaches. A custom admission webhook or controller can implement priority queuing where production hotfix pipelines for the checkout-service bypass the queue while routine feature branch builds for the order-processing-service wait during peak periods. Node affinity and tainting strategies dedicate specific node pools to CI/CD workloads, preventing pipeline pods from competing with production application pods for compute resources. Spot instances or preemptible VMs provide cost-effective burst capacity for the less critical build workloads, with pipeline steps designed to be idempotent so they can safely retry if a spot node is reclaimed.

The Tekton controller itself becomes a scaling bottleneck in large enterprises running thousands of PipelineRuns daily across hundreds of namespaces. The controller is a single reconciliation loop that watches all PipelineRun and TaskRun resources across the cluster. At high volume, the reconciliation queue grows, increasing latency between PipelineRun creation and pod scheduling. Mitigation strategies include deploying multiple Tekton controller replicas with leader election for high availability, increasing controller resource limits and tuning garbage collection intervals for completed runs, partitioning workloads across multiple clusters with a federation layer that routes pipeline requests to the appropriate cluster based on team or workload type, and implementing aggressive pruning of completed TaskRun and PipelineRun resources to reduce the API server watch load on the controller.

Shared infrastructure optimization across tenants significantly reduces the total cost of running Tekton at scale. A cluster-wide container registry mirror caches frequently pulled base images, eliminating redundant pulls across the payments-api, user-auth-service, and inventory-sync pipeline runs. Persistent volume-backed workspace caching stores dependency downloads such as Maven repositories, npm caches, and Go module caches in shared or per-team persistent volumes, reducing build times by 40-60 percent for subsequent runs. Tekton Results, a separate component that offloads completed run data to a database, allows aggressive pruning of in-cluster resources while maintaining the full audit trail needed for compliance. The combination of these optimizations allows enterprise platforms to support thousands of daily pipeline runs with predictable performance, fair resource allocation, and strong security isolation between teams.

The Tekton-ArgoCD integration represents the canonical implementation of the GitOps model where CI and CD are deliberately separated with Git as the single source of truth between them. This separation is not merely architectural elegance but a fundamental security and auditability requirement. The CI system (Tekton) has permissions to access source code repositories, build infrastructure, and artifact registries, while the CD system (ArgoCD) has permissions to deploy to Kubernetes clusters. Neither system needs the other's credentials. The Git repository that sits between them serves as an auditable, version-controlled contract that captures every deployment decision as a git commit with full attribution and rollback capability.

The integration workflow begins when a developer pushes code to the payments-api source repository. A Tekton EventListener receives the webhook from GitHub or GitLab and triggers a PipelineRun. The Tekton pipeline clones the source code, runs unit tests and security scans, builds the container image, pushes it to the registry with a digest-based tag, and then executes the critical handoff step: updating the GitOps deployment repository. This update step clones the deployment repository containing the Kubernetes manifests or Helm values for the payments-api, modifies the image tag to reference the newly built image digest, commits the change with a meaningful message linking back to the source commit and pipeline run, and pushes to the deployment repository. This git commit is the integration point between Tekton and ArgoCD.

ArgoCD is configured with an Application resource that points to the payments-api directory in the GitOps deployment repository. ArgoCD continuously polls the repository or receives webhook notifications when changes are pushed. When ArgoCD detects that the desired state in Git differs from the actual state in the Kubernetes cluster, it performs a sync operation that applies the updated manifests. For the order-processing-service and checkout-service, separate ArgoCD Application resources point to their respective directories in the same or different deployment repositories. ArgoCD's sync policies can be configured for automatic sync with auto-prune and self-heal enabled for development environments, while production environments use manual sync with required approvals through ArgoCD's sync windows and RBAC policies.

The deployment repository structure is critical for this integration to work cleanly at scale. A well-designed GitOps repository uses a directory hierarchy that separates environments and services. The base directory contains the common Kubernetes manifests for each service, while overlay directories for dev, staging, and production contain environment-specific patches using Kustomize. When Tekton updates the image tag, it modifies only the environment-specific overlay, allowing the same pipeline to promote images through environments by targeting different overlay directories. For Helm-based deployments, the deployment repository contains values files per environment, and Tekton updates the image.tag value in the appropriate values file. This structure allows ArgoCD to manage dozens of services across multiple environments from a single repository with clear separation of concerns.

Operational considerations for the Tekton-ArgoCD integration include handling deployment failures, managing secrets, and implementing progressive delivery. When ArgoCD detects a failed deployment through health checks on the newly deployed pods of the inventory-sync service, it can be configured to automatically roll back by reverting the git commit. The Tekton pipeline can include a final notification step that queries the ArgoCD API to verify the sync completed successfully, closing the feedback loop. Secrets management requires careful coordination because the GitOps repository must not contain plaintext secrets. Solutions include Sealed Secrets, External Secrets Operator, or HashiCorp Vault with the ArgoCD Vault plugin, where secret references in the GitOps repository are resolved at deployment time. For progressive delivery of the user-auth-service, ArgoCD integrates with Argo Rollouts to perform canary or blue-green deployments, with Tekton triggering the initial rollout and ArgoCD managing the progressive traffic shifting based on metric analysis from Prometheus.

Tekton Triggers provides a powerful event processing pipeline consisting of EventListeners, Interceptors, TriggerBindings, and TriggerTemplates. While the built-in interceptors for GitHub, GitLab, Bitbucket, and CEL expression filtering handle common scenarios, enterprise environments frequently require custom business logic that cannot be expressed through simple payload filtering. Custom interceptors fill this gap by providing a programmable extension point where arbitrary code can inspect, validate, transform, and enrich incoming webhook payloads before they trigger pipeline runs. This capability transforms Tekton from a simple webhook-to-pipeline mapper into a sophisticated event processing platform.

A custom interceptor is an HTTP service that implements a simple contract: it receives a POST request with the original webhook payload and trigger context in the body, performs its custom logic, and returns a modified payload with optional additional fields in the extensions map. When the checkout-service repository sends a push webhook to the Tekton EventListener, the request passes through the interceptor chain in order. The built-in GitHub interceptor first validates the webhook signature using the shared secret, confirming the request genuinely originated from GitHub. Then the custom interceptor receives the validated payload and executes business-specific logic. For example, it might query the organization's service registry to determine which team owns the checkout-service repository, fetch the deployment configuration from a central configuration management system, check a deployment freeze calendar to determine if builds should proceed or be queued, and enrich the payload with metadata like the service's tier level, required approval gates, and target deployment environments.

Implementing a custom interceptor involves deploying a lightweight HTTP server as a Kubernetes service in the same cluster as the Tekton EventListener. The interceptor service receives the incoming request body containing the original webhook headers, the webhook body, and any extensions added by previous interceptors in the chain. It processes the data and returns a response with an extensions field containing additional key-value pairs that become available to subsequent TriggerBindings. For the payments-api webhook processing, a custom interceptor written in Go or Python might parse the commit messages to extract Jira ticket references, query the Jira API to verify the ticket is in the correct status for deployment, look up the repository in an internal service catalog to determine the build profile such as Java-Maven versus Node-npm versus Go-modules, and add these enrichments as extensions that the TriggerTemplate uses to parameterize the pipeline run with the correct build steps and deployment targets.

The interceptor chain ordering is critical for both security and correctness. The recommended pattern places the built-in platform interceptor first for webhook signature validation, followed by a CEL interceptor for lightweight filtering such as ignoring pushes to documentation-only branches, and then the custom interceptor for business logic enrichment. This ordering ensures that the custom interceptor only processes authenticated, pre-filtered events, reducing the attack surface and processing load. For the order-processing-service and inventory-sync pipelines, the custom interceptor can implement conditional pipeline selection: examining the changed file paths in the push event to determine whether a full build pipeline, a documentation-only pipeline, or a database-migration-only pipeline should be triggered, routing each to a different TriggerTemplate.

Production deployment of custom interceptors requires attention to reliability, observability, and security. The interceptor service should be deployed with multiple replicas behind a Kubernetes service for high availability, because a failing interceptor blocks all pipeline triggers. Health check endpoints allow Kubernetes to detect and replace unhealthy interceptor pods. Structured logging with request tracing enables debugging when webhooks fail to trigger expected pipeline runs. The interceptor should implement timeouts for external API calls to prevent a slow Jira API response from blocking the entire event processing chain. Rate limiting protects the interceptor from webhook storms when the user-auth-service repository receives a burst of push events during a large merge operation. Caching of external API responses such as service catalog lookups and team ownership data reduces latency and external dependency on every webhook invocation.

Pipeline performance directly impacts developer productivity and organizational velocity. When the payments-api pipeline takes 25 minutes to complete, developers context-switch to other work, lose flow state, and batch their commits to avoid frequent feedback delays. Reducing that pipeline to 8 minutes through systematic optimization transforms the development experience from asynchronous batch processing to near-interactive feedback. The optimization strategy spans three dimensions: eliminating redundant work through caching, maximizing concurrency through parallel execution, and right-sizing resources to eliminate bottlenecks without wasting cluster capacity.

Dependency caching is the highest-impact optimization for most pipelines. Without caching, every pipeline run for the order-processing-service downloads hundreds of megabytes of Maven dependencies, npm packages, or Go modules from external registries. A PersistentVolumeClaim-backed workspace preserves the dependency cache across pipeline runs. The first run populates the cache, and subsequent runs resolve dependencies from the local cache in seconds rather than minutes. For Maven-based services, the .m2/repository directory is stored on the PVC. For npm, the node_modules directory or npm cache is persisted. The cache workspace is mounted into each task that needs dependencies, and the pipeline includes a cache validation step that checks cache freshness against the lockfile hash. When the lockfile changes, indicating dependency updates, the cache is invalidated and rebuilt. This single optimization typically reduces pipeline duration by 30-50 percent for dependency-heavy projects like the checkout-service that pulls 400+ npm packages.

Parallel task execution exploits the DAG nature of Tekton pipelines to run independent tasks simultaneously. The runAfter field in pipeline task definitions creates explicit dependencies, and any tasks without dependencies on each other execute concurrently. For the user-auth-service pipeline, the unit test task, lint task, and security scan task can all run in parallel after the clone-and-install task completes, since they share no data dependencies beyond the source code workspace. More aggressive parallelism splits a single large test suite into multiple parallel TaskRuns using a fan-out pattern. The test suite is partitioned by module, package, or time-based balancing, with each partition executing as a separate TaskRun. A final fan-in task collects the results and generates a unified test report. This pattern reduces the inventory-sync service's 12-minute test suite to 4 minutes using three parallel test runners.

Container image build optimization addresses the longest-running step in most pipelines. Layer caching with kaniko or buildkit reuses previously built layers when Dockerfile instructions have not changed. Building with a persistent buildkit daemon rather than ephemeral build containers maintains the layer cache across builds, reducing rebuild times from 5 minutes to under 1 minute for incremental changes. Multi-stage builds eliminate unnecessary tools and dependencies from the final image, reducing push time. For services with multiple components like the payments-api that includes both a Java backend and a React frontend, the backend and frontend builds execute as parallel tasks, with a final task that combines the artifacts into the deployable image.

Resource management optimization ensures that pipeline tasks request appropriate CPU and memory to avoid both throttling and waste. Under-provisioned tasks run slowly due to CPU throttling or get OOM-killed, causing pipeline failures that require full reruns. Over-provisioned tasks waste cluster capacity that could serve other pipeline runs. The optimization process involves running pipelines with resource metrics collection enabled, analyzing actual resource consumption using Prometheus metrics from the kubelet, and adjusting requests and limits to match the observed profile with reasonable headroom. For the order-processing-service pipeline, the compile step might need 2 CPU cores and 4Gi memory for 90 seconds, while the unit test step needs 1 CPU and 2Gi for 3 minutes. Setting these precisely allows the Kubernetes scheduler to bin-pack pipeline pods efficiently, increasing the number of concurrent pipeline runs the cluster can support.

Pipeline-level structural optimizations reduce overhead beyond individual task tuning. Combining multiple small steps into a single step within a task eliminates pod scheduling overhead, since each step in a Tekton task shares the same pod but each task in a pipeline creates a new pod. Pod scheduling adds 5-15 seconds of overhead, so consolidating 6 sequential single-step tasks into 2 multi-step tasks saves 20-60 seconds. Tekton Results offloads completed PipelineRun and TaskRun records from etcd to a database, reducing API server load and improving controller reconciliation speed as the cluster scales to thousands of daily pipeline runs. Finally, workspace volume optimization matters: using emptyDir volumes for ephemeral data and PVCs only for persistent caches avoids the volume provisioning and attachment delays that occur with dynamically provisioned persistent volumes in cloud environments.

Securing Tekton in a multi-team environment requires thinking beyond simple access control lists to a comprehensive security architecture that addresses four distinct threat surfaces: who can define pipelines, what those pipelines can do at runtime, how pipeline credentials are scoped, and what governance policies ensure pipeline definitions comply with organizational standards. A developer on the payments-api team should be able to create and run pipelines in their namespace but must not be able to read secrets from the user-auth-service namespace, modify shared ClusterTasks used by all teams, or run pipeline steps with privileged container access that could compromise the node.

Kubernetes RBAC forms the foundation layer. Custom Roles define the specific Tekton API operations each team can perform. A pipeline-developer Role grants create, get, list, and watch permissions on Pipelines, Tasks, PipelineRuns, and TaskRuns within the team's namespace, along with read access to ClusterTasks and ClusterPipelines that provide shared reusable components. A pipeline-admin Role additionally grants update and delete permissions plus the ability to manage EventListeners and TriggerBindings for webhook integrations. A pipeline-viewer Role provides read-only access for stakeholders who need to monitor pipeline status without modification rights. These Roles are bound to team groups through RoleBindings, with the group membership managed by the organization's identity provider through OIDC integration with the Kubernetes API server.

ServiceAccount isolation is the critical runtime security boundary. Each team's PipelineRuns execute under a team-specific ServiceAccount rather than a shared or default service account. The checkout-service team's ServiceAccount has an ImagePullSecret for their team's registry namespace, a Secret containing their deployment credentials for the staging environment, and RBAC permissions scoped to their namespace. When a PipelineRun executes, the TaskRun pods inherit the ServiceAccount's identity, and any Kubernetes API calls from within the pipeline steps are authorized against that ServiceAccount's RBAC bindings. This prevents a malicious or misconfigured pipeline step from accessing resources outside the team's boundary. Critical secret isolation ensures that the payments-api team's database credentials are stored in their namespace and accessible only to their ServiceAccount, while the order-processing-service team cannot reference those secrets even if they know the secret name.

Policy engines like OPA Gatekeeper or Kyverno provide governance over pipeline definitions that RBAC alone cannot enforce. While RBAC controls who can create a Pipeline resource, policy engines control what that Pipeline can contain. Policies can enforce that all TaskRun steps must use images from an approved internal registry rather than arbitrary public images, that no pipeline step can run as root or with privileged security context, that all PipelineRuns must include a security scanning task before the image build task, that resource requests and limits must be specified for every step to prevent resource exhaustion, and that all pipeline workspaces must use volumeClaimTemplates rather than hostPath volumes. These policies are evaluated at admission time when the Pipeline or PipelineRun resource is created, rejecting non-compliant definitions with a descriptive error message before they ever execute.

Audit logging and monitoring complete the security architecture. Kubernetes audit logs capture every Tekton API operation with the authenticated identity, providing a compliance-ready trail of who created, modified, or triggered each pipeline. Prometheus metrics from the Tekton controller track pipeline execution patterns, and alerting rules flag anomalies such as pipeline runs executing at unusual hours, pipelines accessing resources outside their normal scope patterns, or sudden increases in failed pipeline runs that might indicate credential compromise. For the inventory-sync service pipeline, audit policies can be configured to log every access to production deployment credentials at the RequestResponse level, creating a complete record of when production deployments were triggered and by whom. Integration with SIEM platforms like Splunk or Elastic Security enables correlation of Tekton activity with broader infrastructure security events, detecting scenarios where a compromised developer account attempts to modify pipeline definitions to exfiltrate secrets through build step output logs.

Pipeline-as-code is the practice of storing CI/CD pipeline definitions in the same version-controlled repository as the application source code, treating them as first-class software artifacts subject to the same review, testing, and versioning processes as the application itself. Tekton's pipeline-as-code implementation, through the Pipelines-as-Code component, extends this concept by providing a controller that automatically discovers, validates, and executes pipeline definitions from repository directories when triggered by Git events. This eliminates the disconnect between the application code and the pipeline that builds it, a problem that plagues organizations where pipeline definitions live in a separate central repository and drift out of sync with the applications they serve.

The Pipelines-as-Code controller integrates with GitHub, GitLab, and Bitbucket through webhook events and GitHub App installations. When a developer on the payments-api team creates a pull request, the controller detects the .tekton/ directory in the repository, reads the PipelineRun definitions from that directory, resolves any Task references using in-repo definitions or Tekton Hub references, and creates the PipelineRun in the target namespace. The pipeline runs against the pull request branch, and the results are reported back to the Git provider as commit status checks. This means the pipeline definition in the pull request branch is the one that executes, allowing developers to modify both their application code and the pipeline in the same pull request and see the combined effect immediately. For the checkout-service team, updating a Dockerfile and simultaneously modifying the build pipeline to accommodate the new multi-stage build structure happens in a single reviewable, testable pull request.

Version control of pipeline definitions provides capabilities that centrally managed pipelines cannot match. Every pipeline change has a Git commit with an author, timestamp, reviewer, and description of why the change was made. If the order-processing-service pipeline starts failing after a recent change, git log on the .tekton/ directory immediately reveals what changed, who changed it, and the associated code review discussion. Rolling back a pipeline change is a standard git revert. Branching allows the inventory-sync team to experiment with pipeline changes on a feature branch without affecting the main pipeline. Release tagging pins a specific pipeline version to a release, ensuring reproducible builds even months later. This versioning capability is essential for regulated environments where auditors need to prove that the pipeline used to build a specific release artifact was reviewed and approved through the same change management process as the application code.

The organizational pattern for pipeline-as-code at scale involves a layered template system. A central platform-pipelines repository maintained by the DevOps team contains base Task definitions and Pipeline templates for common technology stacks like Java-Maven, Node-npm, Python-pip, and Go-modules. Application teams reference these shared tasks from their .tekton/ PipelineRun definitions using Tekton Bundles, which package Tasks and Pipelines as OCI artifacts versioned in the container registry. The user-auth-service team's .tekton/pull-request.yaml references the shared maven-build task from the latest approved bundle version, adds service-specific integration test tasks defined locally in the repository, and customizes parameters like the deployment target and notification channels. When the platform team releases a new version of the shared tasks with security improvements, application teams update the bundle reference in their .tekton/ definitions through standard pull requests, ensuring controlled adoption with review and testing.

Operational considerations for pipeline-as-code include security boundaries, resource management, and debugging workflows. The Pipelines-as-Code controller must validate that pipeline definitions in untrusted pull requests from external contributors do not reference privileged resources or attempt to exfiltrate secrets. The controller supports a policy where pull requests from non-collaborators require an explicit comment like /ok-to-test from a repository maintainer before the pipeline executes, preventing untrusted code from running in the CI environment. The .tekton/ directory can contain multiple PipelineRun definitions for different events: pull-request.yaml for pull request validation, push-main.yaml for main branch builds, and release.yaml for tagged releases. Each definition specifies its trigger conditions through annotations, allowing fine-grained control over which pipeline runs for which event. Debugging pipeline failures uses the standard Tekton tools like tkn pipelinerun logs with the addition of being able to reproduce any historical pipeline run by checking out the corresponding Git commit and applying the .tekton/ definitions manually.

Migrating from Jenkins to Tekton is one of the most common enterprise CI/CD modernization efforts, and it is far more complex than a syntax translation exercise. Jenkins represents a fundamentally different paradigm: a long-running stateful server with an imperative scripting model, extensive plugin ecosystem, and decades of accumulated organizational knowledge embedded in Groovy shared libraries and job configurations. Tekton represents the cloud-native paradigm: stateless, declarative, Kubernetes-native, with each pipeline run as an isolated pod lifecycle. The migration requires not just rewriting pipelines but rethinking how the organization approaches CI/CD, retraining teams, and establishing new operational patterns. Organizations that attempt a big-bang migration by converting all Jenkins pipelines simultaneously invariably fail. The successful pattern is incremental migration with parallel operation.

The conceptual mapping between Jenkins and Tekton forms the foundation of migration planning. A Jenkins Declarative Pipeline with stages maps to a Tekton Pipeline with Tasks. Each stage becomes a Task, and the stage's steps become the Task's steps. A Jenkins Scripted Pipeline's arbitrary Groovy logic requires decomposition into discrete Tasks with well-defined inputs and outputs. Jenkins shared libraries, which encapsulate reusable pipeline logic across the payments-api, order-processing-service, and checkout-service, map to Tekton ClusterTasks or Tasks packaged as Tekton Bundles in an OCI registry. Jenkins credentials stored in the credential manager map to Kubernetes Secrets referenced by ServiceAccounts or mounted as workspace volumes. Jenkins agent labels that route builds to specific node types map to Tekton TaskRun nodeSelector or affinity rules. Jenkins parameterized builds map to Tekton Pipeline params. Jenkins post-build actions and notifications map to Tekton finally tasks that execute regardless of pipeline success or failure.

The incremental migration strategy begins with an assessment phase where all Jenkins jobs are catalogued by complexity, frequency, and business criticality. Simple jobs like the inventory-sync nightly build with straightforward clone-test-build-deploy stages are migrated first to build team experience. Complex jobs like the payments-api multi-branch pipeline with 15 stages, custom Groovy logic, parallel test execution, and integration test environments are migrated last after the team has established Tekton patterns and reusable task libraries. During the parallel operation phase, both Jenkins and Tekton run simultaneously. A routing layer, implemented as a webhook proxy or Git-provider webhook configuration, directs events for migrated repositories to Tekton EventListeners while unmigrated repositories continue triggering Jenkins. This allows team-by-team migration at each team's pace rather than a forced cutover.

The most challenging aspect of Jenkins-to-Tekton migration is replacing Jenkins plugins with container-based equivalents. Jenkins has over 1800 plugins covering every conceivable integration: SonarQube analysis, Slack notifications, Jira ticket updates, AWS deployments, Artifactory publishing, and hundreds more. Each plugin used in the organization's Jenkins pipelines must be replaced with either a Tekton Catalog task from the Tekton Hub, a custom Task that runs the equivalent CLI tool in a container image, or a purpose-built container image that packages the tool with the required configuration. For example, the Jenkins SonarQube plugin that the user-auth-service pipeline uses becomes a Tekton Task running the sonar-scanner CLI in the sonarsource/sonar-scanner-cli container image. The Jenkins Slack notification plugin becomes a Tekton Task that calls the Slack API directly using curl or a lightweight notification container. This container-based approach is actually more transparent and portable than Jenkins plugins, but the initial migration effort for organizations using 30-50 plugins is substantial.

Post-migration operational differences require organizational adaptation. Jenkins provides a centralized web UI where managers and release engineers monitor all pipeline activity across the organization. Tekton's native interface is the Kubernetes API accessed through kubectl or the tkn CLI, which is less accessible to non-technical stakeholders. The Tekton Dashboard or OpenShift Pipelines console provides a web UI, but it is simpler than Jenkins and may require customization to match the organization's monitoring needs. Jenkins's build history and artifact archival are built-in features, while Tekton requires Tekton Results for long-term run history and explicit artifact publishing to registries or storage systems. Team training must address not just Tekton syntax but Kubernetes concepts like pods, services, persistent volumes, and RBAC that are prerequisites for understanding how Tekton works. Organizations that underinvest in training find that teams revert to writing shell scripts inside Tekton steps that recreate Jenkins patterns rather than embracing the declarative cloud-native model.