Kiro for Data Engineers

Data work is a good fit for Kiro for one specific reason: data mistakes fail quietly. A wrong join does not throw an exception, it produces plausible numbers, and nobody notices until a dashboard is wrong in a meeting.

That is exactly the failure mode a written plan and a committed set of conventions are good at preventing. This chapter is the data practitioner’s path through Kiro: pipelines, notebooks, dataset documentation, transformation tests, database access, and the safety settings that matter when your repository sits next to real data.

The data practitioner path

If you came here to learn Kiro for data work rather than application development, this is the route:

1. What is Kiro? for the shape of the tool.
2. Core Concepts for the vocabulary: specs, steering, hooks, MCP, agents, skills, powers.
3. This chapter for how those apply to pipelines and analysis.
4. The CLI Guide, because a lot of data work happens on machines with no desktop.
5. Choosing a Model, because bulk work is where model choice actually shows up on the bill.

The home page lists this as Path 3, the data practitioner path. You can skip the application-development chapters entirely and lose nothing.

Steering: your warehouse conventions, written once

This is the highest-value thing a data team can do in the first hour, and it is a Markdown file.

Data code is convention-heavy. Layer names, column casing, surrogate key rules, partitioning, what counts as “active”, which date column is authoritative. An agent that does not know those rules will invent reasonable-looking alternatives, and reasonable-looking alternatives are exactly what nobody catches in review.

A warehouse.md steering file in .kiro/steering/:

# Warehouse conventions

- Layers: `stg_` reads raw source, `int_` holds intermediate logic,
  `fct_` and `dim_` are the only layers dashboards may query.
  Why: dashboards broke twice when they read intermediates directly.
- Columns are snake_case. Timestamps end in `_at` and are UTC.
- Every fact table is partitioned on `event_date`. Why: our queries
  are almost always date-ranged, and full scans cost real money.
- Never use `SELECT *` in a model. Column drift used to break
  downstream models silently.
- "Active customer" means an order in the last 90 days. This
  definition is used by finance; do not redefine it locally.

Note the why on each rule. Steering works better when the agent understands the reason, because it can then apply the rule to cases you did not list.

Keep one domain per file. A sql-style.md for formatting, a warehouse.md for structure, a metrics.md for definitions. Use the fileMatch inclusion mode so your SQL rules load when models/**/*.sql is in play and stay out of unrelated conversations.

Never put credentials in steering. It is committed to your repository.

A spec for a new ingestion job

Pipeline changes are the textbook case for a spec: risky, hard to undo, and other people depend on the output. Here is what the three files look like for a generic new ingestion job.

requirements.md, in EARS Easy Approach to Requirements Syntax: structured requirement sentences like WHEN … THE SYSTEM SHALL … that can each become a test. notation, so every line is testable:

WHEN a new source file lands in the landing bucket
THE SYSTEM SHALL load it into `stg_orders` within 15 minutes

WHEN a record fails schema validation
THE SYSTEM SHALL route it to `stg_orders_quarantine` with the
failure reason and SHALL NOT stop the run

IF the same source file is processed twice
THEN THE SYSTEM SHALL produce no duplicate rows in `stg_orders`

WHILE a backfill is running
THE SYSTEM SHALL keep serving the previous partition unchanged

Read those slowly. Every one of them is a decision somebody would otherwise make silently at 4pm on a Thursday: the freshness target, what happens to bad records, whether the load is idempotent, and whether backfills are visible to readers.

design.md captures what reviewers actually argue about: the schema, the idempotency mechanism, the backfill strategy, ordering guarantees, and the partitioning choice.

tasks.md sequences the work so it is safe: create the table, load into a shadow partition, verify counts, cut over, then remove the old path. Independent tasks run concurrently in waves, so keep them decoupled.

Run the optional requirements analysis before approving. It flags ambiguity, contradictions, and gaps, and on a pipeline spec it usually finds at least one requirement that two people were reading differently.

Notebooks and analysis projects

To Kiro, a notebook is a file in the repository. The agent can read it, refactor a sprawling cell into a tested function, or draft the writeup from your results files.

Two habits make this work. Keep a steering file with your data dictionary and metric definitions, so every summary uses the same terms; nothing makes an analysis harder to trust than three documents defining “session” three ways. And point at results rather than pasting them: /context add "experiments/**" in the CLI, or #folder in the IDE, gives the agent current files instead of a stale snapshot.

A remote flow, for the common case where the work lives on a training box:

1. SSH in and run kiro-cli.
2. /context add "experiments/**" to pull in the results.
3. “Summarise this week’s runs into experiments.md: a table of run, parameters, headline metric, and verdict.”
4. !ls runs/ for quick shell checks without leaving the session.

One honest limit: the agent drafts the writeup, but it cannot validate your science. Recompute any headline number yourself before someone makes a decision on it.

Documenting datasets

Dataset documentation is the chore that always loses, because the cost of skipping it lands on someone else three months later.

Three things help. The spec files themselves are living documentation, because requirements.md records what a dataset is supposed to mean, in the repository next to the code rather than in a wiki that drifts. Steering captures the gotchas new teammates always trip over. And a hook can refresh the dataset doc whenever the matching model changes, which attacks the real problem: docs going stale silently. Recipe: update docs on save.

Scope doc hooks to narrow folders. A hook firing on every save in the whole repository turns useful automation into background noise.

Testing transformations

Write the invariants as acceptance criteria, so they are testable statements rather than wishes:

• Row counts in equal row counts out, or the filter is explicit.
• No nulls in key columns after the join.
• Totals reconcile with the source within a stated tolerance.
• The same input processed twice produces the same output.

Property-based testing suits data work particularly well, because properties assert over whole distributions of generated inputs rather than a handful of hand-picked rows. A hook that runs the test suite on save closes the loop.

The trap to avoid: asking the agent to write tests from the code. It will infer intent from what the code does, so if the transformation is wrong, the test will assert the wrong behaviour and pass forever. Write tests from the requirements.

Connecting to a database with MCP

If your database, warehouse, or catalogue exposes an MCP server, the agent can inspect real schemas instead of guessing column names. That single change removes most hallucinated-column bugs.

Configure it in .kiro/settings/mcp.json for the project or ~/.kiro/settings/mcp.json for yourself, keeping credentials in environment variables:

{
  "mcpServers": {
    "warehouse": {
      "command": "npx",
      "args": ["-y", "your-database-mcp-server"],
      "env": { "DB_URL": "${DB_URL}" },
      "autoApprove": [],
      "disabled": false
    }
  }
}

This site does not maintain a verified list of database MCP servers, so use whichever your vendor or community publishes and read it before you install it. Full walkthrough: add an MCP server.

Keeping data out of the agent’s reach

Data repositories collect things the agent should never read: a dump someone took for debugging, an export from a support ticket, a credentials file that should not exist but does.

A .kiroignore file at the repository root, in gitignore syntax:

*.sql
*.dump
*.csv
customer-data/
exports/
.env

Note the trade-off in that first line. If your SQL models are .sql files you want the agent to read, do not exclude *.sql wholesale; exclude the dump directory instead. Ignore rules cannot re-include a file inside an excluded directory, so put the exclusions at the right level from the start.

A global version lives at ~/.kiro/settings/kiroignore, which is the right place for patterns you want on every project. Support differs by surface: full in the IDE, applied to search results in CLI V3, and not yet available in Web or Mobile. So .kiroignore is a good hygiene layer, not a security boundary. Recipe: .kiroignore for secrets.

Remote machines and scheduled work

Two situations come up constantly in data teams.

A machine with no desktop. Kiro CLI runs over SSH with the full primitive set, and it reads the same .kiro/ folder as everyone else’s IDE, so conventions travel with the repository. Alternatively, start a cloud session with kiro-cli --cloud and let the agent run in a managed AWS sandbox instead of on a box you are sharing.

A chore that should happen on a schedule. The CLI runs headless in CI: set KIRO_API_KEY from your secret store, run kiro-cli chat --no-interactive "…", and restrict tools with --trust-tools=read,grep for a report-only job. Kiro Web Automations do the same thing from the browser on Pro and above, on an hourly, daily, or CRON schedule, with a pull request as the output.

Start any unattended data job read-only. Have it report rather than fix, until you trust it.

Cheap models for bulk work

This is the lever most data teams miss. Kiro’s models carry credit multipliers relative to the Auto router at 1.0x, and the spread across the lineup is more than twenty to one. Backfilling docstrings across two hundred models, or drafting column descriptions from a schema, is bulk pattern work that a low-multiplier model handles fine.

You can also lower reasoning effort to low or medium for mechanical jobs. Effort is set per session, workspace, or user, and lower effort means faster answers and fewer credits.

Keep the frontier models for the work that deserves them: designing a schema migration, reasoning about idempotency, or reviewing a transformation you cannot afford to get wrong.

Nine broader jobs, including the non-data ones, are in Use Cases. The traps worth knowing before you point an agent at production data are in Common Mistakes.