ADF migration, run by an agent fleet.

What the Documenter reads in Azure Data Factory

Azure Data Factory (ADF) presents a clean visual surface. The logic that actually runs lives in the JSON beneath it, and in the runtime bindings the canvas never shows. The Documenter (the agent that builds the knowledge base before any rebuild begins) reads that logic, not just the labels. It parses the activity graph, resolves the expression chains, and follows the connection and compute bindings to the edge of the estate, then hands back a reviewable inventory a human signs off at the assessment gate. Autonomous in the loop. Accountable at the gates.

The JSON artifacts, read as ARM resources

When a factory is connected to Azure Repos Git or GitHub, every ADF artifact is persisted as a JSON file (one file per resource) conforming to Azure Resource Manager (ARM) template conventions. The Documenter reads the repo, not the live service, because the repo is the reproducible source of truth.

A pipeline file carries a name and a properties object holding activities[], parameters, concurrency, and annotations. Inside activities[] the Documenter separates the two kinds of step. Execution activities (Copy and transform activities such as data flow, stored procedure, and notebook) carry typeProperties, an optional linkedServiceName, a policy (defaults matter when reconstructing behavior: 12-hour timeout, zero retries, a 30-second retry interval, secureOutput), and dependsOn. Control activities (If Condition, ForEach, Until, Switch, Execute Pipeline) carry typeProperties and dependsOn but no activity-level linked service. Because a Git-backed factory exports datasets and linked services as their own files, the Documenter stitches the graph back together by resolving each referenceName.

Reconstructing execution order from dependsOn

dependsOn is ADF’s dependency graph and the analogue of SSIS precedence constraints. Each dependency names a prior activity and one or more dependency conditions: Succeeded, Failed, Skipped, or Completed. The Documenter reads these to rebuild not just the happy path but the error-handling branches: a Failed edge is a deliberate recovery route, and a Skipped edge propagates when an upstream activity never ran. Activities with no dependency between them may run in parallel, so concurrency is recorded as a property of the graph, not an accident of layout. The diagram shows boxes and arrows; the JSON tells you which arrow fires under which outcome.

Resolving expressions and parameters

ADF’s @-prefixed expression language is where metadata-driven behavior hides. Parameters surface as @pipeline().parameters.<name> and @dataset().parameters.<name>, and activity results chain through @activity('Name').output. Because datasets, linked services, pipelines, and data flows can all be parameterized, one generic pipeline can stand in for dozens of concrete runs. The Documenter resolves these chains to describe what the pipeline does; where the effective value depends on a control table or runtime input, it records the dependency rather than inventing a value. Embedded dynamic content and string interpolation are treated the same way: logic to be parsed and re-expressed, not free text.

Connections and compute: linked services, datasets, and integration runtimes

Linked services are the connection definitions, and datasets are the typed, parameterizable views over the data they reach. The Documenter flags a structural quirk: non-Key-Vault linked services publish immediately because their credentials must be encrypted, a frequent cause of Git/live-mode drift.

Compute is the Integration Runtime (IR), and its type determines what can move. The Azure IR is managed serverless compute for cloud-to-cloud copies, mapping data flows, and activity dispatch. The self-hosted IR (SHIR) is software installed inside a private or on-premises network, required to reach restricted stores or use custom drivers. The Azure-SSIS IR is a managed VM cluster that runs only lifted .dtsx packages. Every SHIR binding is an on-premises dependency that will not migrate on its own; every Azure-SSIS package is logged as a rebuild candidate, never a runtime to claim on the target.

Triggers

The Documenter also records what starts a pipeline. Schedule triggers fire on a wall-clock cadence, many-to-many with pipelines. Tumbling window triggers run fixed-size, non-overlapping intervals, retain state, and support backfill, exposing windowStartTime and windowEndTime. Event-based triggers fire on storage events or Event Grid custom events. Capturing the trigger family alongside the activity graph lets the rebuild reproduce the cadence and the backfill semantics on the target.