# Technical Documentation: Azure/azureml-examples > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/Azure/azureml-examples) ### Overview (README.md) # Azure Machine Learning examples [](LICENSE) Welcome to the Azure Machine Learning examples repository! ## About This Repository The azureml-examples repository contains examples and tutorials to help you learn how to use Azure Machine Learning (Azure ML) services and features. ### Getting Started If you're getting started with Azure ML, consider working through our [tutorials] for the v2 Python SDK. You may also want to read through our [documentation](#supplementary-documentation). ### SDKs The `sdk/` folder houses the examples for the Azure ML SDKs across several languages. We have an extensive collection of examples for the [Azure ML Python SDK v2][azure cli ml extension v2 overview] in [`sdk/python`][azureml python sdk v2 examples]. We also offer some examples for our SDKs in other languages: * .NET: [`sdk/dotnet`][azureml dotnet sdk v2 examples] * TypeScript: [`sdk/typescript`][azureml typescript sdk v2 examples] ### Azure Machine Learning extension for Azure CLI The [`cli/` folder][azureml cli extension examples] hosts our examples to use the [Azure Machine Learning extension][azure cli ml extension v2 overview] for [Azure CLI][azure cli overview]. _Note_: If you're looking for examples that submit Azure ML jobs that run non-Python code, see: * **R**: [`cli/jobs/single-step/r`](./cli/jobs/single-step/r) ## Supplementary Documentation - [Azure Machine Learning Documentation](https://docs.microsoft.com/azure/machine-learning) - [AzureML Python SDK v2 Overview] - [Azure CLI ML extension v2 Overview] ## Contributing We welcome contributions and suggestions! Please see the [contributing guidelines](CONTRIBUTING.md) for details. ## Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). Please see the [code of conduct](CODE_OF_CONDUCT.md) for details. [tutorials]: ./tutorials [azure cli overview]: https://learn.microsoft.com/en-us/cli/azure/what-is-azure-cli [azureml cli extension examples]: ./cli [azureml python sdk v2 examples]: ./sdk/python [azureml dotnet sdk v2 examples]: ./sdk/dotnet [azureml typescript sdk v2 examples]: ./sdk/typescript [azure cli ml extension v2 overview]: https://learn.microsoft.com/en-us/azure/machine-learning/concept-v2?view=azureml-api-2#azure-machine-learning-cli-v2 [azureml python sdk v2 overview]: https://learn.microsoft.com/en-us/azure/machine-learning/concept-v2?view=azureml-api-2#azure-machine-learning-python-sdk-v2 --- ### Sdk Python Generative Ai Rag Code First Docs Mlindex (`sdk/python/generative-ai/rag/code_first/docs/mlindex.md`) # MLIndex An example MLIndex file: ```yaml embeddings: api_base: https://azureml-rag-oai.openai.azure.com api_type: azure api_version: 2023-03-15-preview batch_size: "1" connection: id: Default_AzureOpenAI connection_type: environment deployment: text-embedding-ada-002 dimension: 1536 kind: open_ai model: text-embedding-ada-002 schema_version: "2" index: api_version: 2023-07-01-preview connection: id: /subs//rgs//wss//conns/ connection_type: workspace_connection endpoint: https://azureml-rag-acs.search.windows.net engine: azure-sdk field_mapping: content: content embedding: content_vector_open_ai filename: sourcefile metadata: meta_json_string title: title url: sourcepage index: azure-docs-aoai-embeddings-rcts kind: acs ``` --- ### Sdk Python Jobs Migration Aml Command Job To Foundry Docs AML COMMAND JOB MIGRATION (`sdk/python/jobs/migration/aml-command-job-to-foundry/docs/AML_COMMAND_JOB_MIGRATION.md`) # AML command-job migration to Foundry compute `aml-foundry-migrate` recreates an Azure Machine Learning command job on Foundry compute. It migrates the job definition and, by default, the bytes behind its code, data, and model dependencies. Dataset inputs can instead remain in source storage through the optional zero-copy reference mode. Authentication comes from the active Azure CLI login. This is a Python CLI, not a PowerShell implementation. On Windows it is normally invoked from PowerShell; `scripts/bootstrap_local_env.ps1` only prepares the Python environment. Pipeline, component, sweep, and AutoML jobs are intentionally outside this workflow. The source and target are both single command jobs. ## End-to-end exercise The `exercise` command proves the complete path rather than only translating a JSON document. It performs these operations: 1. Creates local code, folder data, file data, an MLTable, and a custom model. 2. Registers the data and model assets in the source AML workspace. 3. Runs an AML command job on the selected AML compute. 4. Downloads the immutable AML code snapshot and registered model. 5. Runs one small AML export job for all data-like dependencies. 6. Uploads code and data through Foundry Dataset V3 using the project's standard storage connection. 7. Uploads and registers the model through Foundry Model V3. 8. Translates and submits the command job to Foundry compute. 9. Waits for the Foundry job to finish. 10. Downloads the Foundry folder, file, and MLTable outputs, compares their JSON content, and verifies the model output exists in Model V3. 11. Downloads AML and Foundry user-log evidence, extracts canonical application records, and requires the normalized logs and all four outputs to match. ```powershell cd tools\foundrytrainingjob .\scripts\bootstrap_local_env.ps1 az login az account set --subscription aml-foundry-migrate exercise ` --source-subscription ` --source-resource-group ` --source-workspace ` --source-export-compute ` --project-endpoint https://.services.ai.azure.com ` --project-name ` --storage-connection ` --foundry-compute-id /subscriptions//resourceGroups//providers/Microsoft.CognitiveServices/accounts//computes/ ` --foundry-instance-type Singularity.D4_v3 ` --target-job-tier Premium ` --user-assigned-identity-id /subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/ ``` The project endpoint, project name, storage connection, Foundry compute ID, instance type, and API version default to the values already configured by the `foundrytrainingjob` tool. Source workspace arguments are required unless these environment variables are set: - `AML_MIGRATION_SOURCE_SUBSCRIPTION` - `AML_MIGRATION_SOURCE_RESOURCE_GROUP` - `AML_MIGRATION_SOURCE_WORKSPACE` - `AML_MIGRATION_SOURCE_COMPUTE` - `AML_MIGRATION_SOURCE_IDENTITY_DATASTORE` (defaults to `foundrymigrationidentityblob`) - `AML_MIGRATION_DATASET_TRANSFER_MODE` (`upload` or `reference`) - `AML_MIGRATION_SOURCE_STORAGE_CONNECTION` (required for `reference`) ## Live validation boundary This sample does not ship a separate live pytest. Use the explicit `aml-foundry-migrate exercise` command above for the supported live workflow and retain its work directory as evidence. The default `tests/unit` lane mocks cloud boundaries and does not create Azure resources. ## Cost-bounded release qualification The release gate combines every mutually compatible core behavior into one AML source fixture, then migrates that same completed source twice: - `upload`: URI-folder, URI-file, and MLTable bytes are copied into Foundry project Dataset V3 assets. - `reference`: those same three AML inputs become Dataset V3 references through a Foundry connection whose target must match the AML source storage host. Dataset V3 validates URI-folder references at the storage-container boundary. For an AML data asset backed by a subfolder, migration registers the existing container as the reference and appends the original relative blob prefix to every command and templated-environment binding. The path visible to user code therefore remains the AML asset folder; no bytes are copied. URI-file inputs continue to reference their exact source blob. The rich source job contains string, integer, number, and boolean literals; URI-folder, URI-file, MLTable, and custom-model inputs; static and input-templated environment variables; code, environment, compute, identity, shared memory, and timeout behavior; and URI-folder, URI-file, MLTable, and custom-model outputs. Both target runs must reproduce the AML application-log record and all four logical output digests. Foundry currently rejects the native `MLTable` output discriminator during CommonRuntime output registration. Migration therefore preserves the output's `MLTable` definition and payload files but registers the target as a `uri_folder`. Analysis and release evidence report this as an adapted output, not native MLTable asset-type parity. ```powershell aml-foundry-migrate qualify-release ` --source-subscription ` --source-resource-group ` --source-workspace ` --source-export-compute ` --source-storage-connection ` --project-endpoint https://.services.ai.azure.com ` --project-name ` --storage-connection ` --foundry-compute-id ` --foundry-instance-type ` --user-assigned-identity-id ` --work-dir D:\migration-evidence\release-001 ``` The worst case is four cloud jobs: one AML source job, one batched AML export, and two Foundry target jobs. Pass `--existing-source-job` plus `--existing-source-asset-version` to reuse a completed rich source fixture and reduce that to three. Upload and reference target assets always receive distinct versions, so the reference run cannot overwrite the copied datasets. The evidence root contains per-mode `equivalence-report.json`, migration manifests, analysis reports, downloaded raw logs/outputs, and one aggregate `release-validation-report.json`. The aggregate report hashes every retained artifact and each migration implementation file. It fails closed on terminal status, parity, Dataset V3 type/wiring, connection, or runtime-RBAC mismatches. A pass means the report's `qualifiedScope` is ready for a constrained preview. It does not mean unrestricted customer release: `excludedScope` names pipeline, sweep, AutoML, distributed, alternate delivery-mode, alternate model-format, environment-build/private-registry, interactive-service, and advanced MLTable or component-contract variants that still need their own live parity evidence. ## Component-level validation This sample does not ship a component-by-component live matrix. Component behavior is covered at unit level under `tests/unit`; use `exercise` or `qualify-release` for the supported aggregate live workflows. Variants listed in `excludedScope` remain unqualified until retained live parity evidence exists. ## Log and output equivalence Raw AML and Foundry logs are not expected to be byte-identical. Each platform adds different timestamps, rank prefixes, bootstrap output, mount paths, and runtime diagnostics. The fixture therefore emits one stable line: ```text MIGRATION_FIXTURE_RECORD:{"enabled":true,"epochs":3,...} ``` The monitor downloads and retains the raw bundles, extracts every prefixed JSON record, canonicalizes keys and whitespace, removes duplicate records, and then compares SHA-256 digests. A missing, malformed, or different record fails the test. `MIGRATION_FIXTURE_COMPLETED` must also be present on both sides. The report also records each service's terminal status, raw log filenames, and raw-file SHA-256 hashes. Those raw hashes are evidence pointers, not equality assertions: platform wrappers are expected to differ. Output paths and registered asset IDs necessarily differ, so comparison is by logical output rather than physical path: - `results/result.json` - `summary/summary.json` - `metrics_table/metrics.jsonl` - `trained_model/model.json` JSON is parsed and serialized canonically before digesting. The AML output is the oracle, including AML's observed MLTable materialization. Foundry validation first checks its content against that AML result and then the final equivalence step compares every logical output again. Raw logs, raw outputs, normalized values, and both digest sets are recorded in the report. ## Migrate an existing job ### Analyze before writing Run the read-only analyzer before creating any Foundry assets or jobs: ```powershell aml-foundry-migrate analyze ` --source-subscription ` --source-resource-group ` --source-workspace ` --source-job ` --analysis-policy migratable ` --report-file .\migration-analysis.json ``` The analyzer reads AML job/asset metadata and Foundry connection metadata only. It does not need source export compute and does not create assets, jobs, connections, or role assignments. Every capability record includes: - `support`: `supported`, `conditional`, `manual_action`, or `unsupported` - `semanticFidelity`: `equivalent`, `adapted`, `lossy`, or `unknown` - `selectedAction`: for example `translate`, `reference`, `copy`, `replace`, `transform`, `allocate`, `filter`, or `drop` - `verification`: `live`, `unit`, `none`, or `not_applicable` - `blocking`, `externalDependency`, `referenceable`, source/target values, and a concrete remediation when one is known Choose an enforceable policy: | Policy | Pass condition | | --- | --- | | `advisory` | Always exits `0`; inspect the report. | | `migratable` | The concrete invocation has no unsupported or unresolved blockers. | | `lossless` | Migratable, with no lossy or unknown source semantics. | | `reference-only` | Migratable and every external dependency can remain in place. | | `strict` | Lossless and every encountered capability has completed live verification. | A policy failure still writes/prints the report but exits `2`. Operational or authentication errors exit `1`. Use the same guard inline with migration: ```powershell aml-foundry-migrate migrate ` --source-resource-group ` --source-workspace ` --source-export-compute ` --source-job ` --preflight-policy migratable ``` When preflight fails, migration never starts. For `reference-only`, the report also validates that the supplied Foundry storage connection target host matches each resolved AML data URI; an unrelated but valid storage connection does not count. ### Permission and RBAC preflight Analysis also performs a read-only runtime-identity/RBAC inspection. It resolves the target UAI to its principal/client IDs, enumerates effective role assignments including ancestor scopes, and evaluates only roles that provide the required runtime data actions. Generic ARM `Contributor` is **not** treated as blob data access. The required checks are derived from the selected migration strategy: | Scope | Required accepted roles | | --- | --- | | Foundry project | `Foundry User`, `Azure AI Developer`, or `Azure AI Administrator` | | Foundry project storage | `Storage Blob Data Contributor` or `Storage Blob Data Owner` | | Each AML source storage account in reference mode | `Storage Blob Data Reader`, `Storage Blob Data Contributor`, or `Storage Blob Data Owner` | | Private environment ACR | `AcrPull`, `AcrPush`, or the ABAC repository Reader/Writer/Contributor roles | Connection metadata supplies project and target-storage ARM scopes. Resolved AML blob hosts supply source-storage scopes. Private ACR hosts are resolved to registry resources in candidate source/target subscriptions. SAS/API-key connections do not require UAI storage RBAC; AAD/managed-identity connections do. Permission findings use four statuses: - `satisfied`: an unconditional accepted assignment exists at the scope or an ancestor - `conditional`: a matching assignment has an Azure RBAC condition; evaluate the condition manually - `missing`: role assignments were readable, but no accepted assignment exists - `unknown`: the identity/scope/role assignments could not be inspected, or a custom role's actions were not resolved `missing` and `unknown` block `migratable` preflight. Inspection failure is never misreported as a missing assignment. Reports include accepted roles, principal ID, effective assignment scopes, conditions, and remediation. Summary fields include `runtimePermissionsSatisfied`, `permissionCountsByStatus`, `missingPermissionIds`, `unknownPermissionIds`, and `conditionalPermissionIds`. #### Opt-in source-storage access grant The `migrate` command can repair analyzer-confirmed missing source-storage read access before a reference migration: ```powershell aml-foundry-migrate migrate ` --source-resource-group ` --source-workspace ` --source-export-compute ` --source-job ` --dataset-transfer-mode reference ` --source-storage-connection ` --user-assigned-identity-id ` --grant-reference-storage-access ``` This is an explicit RBAC mutation and is disabled by default. The flag: - requires reference transfer and an explicit target job UAI - requires that UAI to already be attached to the Foundry project - runs `migratable` analysis before changing RBAC - makes no change if any non-grantable migration blocker is present - creates only `Storage Blob Data Reader` assignments at exact source storage- account scopes reported as `missing` - does not override `unknown` or conditional findings - uses deterministic role-assignment IDs so retries are idempotent - reruns preflight after the assignment and before creating migration assets or submitting the Foundry job - records created assignments under `referenceStorageRoleAssignments` in CLI output and atomically in `reference-storage-role-assignments.json` before post-grant preflight or migration starts It never creates the Foundry source-storage connection and never grants Foundry project, target-storage, private-ACR, owner, contributor, or role-administration access. It also does not attach identities to the Foundry project. The signed-in caller must already have `Microsoft.Authorization/roleAssignments/write` on each exact source storage scope. Azure RBAC propagation can still delay runtime access; the post-grant preflight proves control-plane visibility, while the migrated job remains the data-plane authority. Static RBAC inspection cannot prove deny assignments, inactive PIM eligibility, custom-role `actions`/`dataActions`, network ACLs, private DNS, service-side connection policy, or eventual role propagation. It also does not prove the **caller** can create AML export jobs or Foundry assets/jobs; source job/asset and connection reads are exercised by analysis, while write authorization is only fully proven by migration or a service-provided validate endpoint. Runtime E2E tests remain the final authority for identity and networking. The formal fixture requires project and target-storage checks in both modes and source-storage reader access in reference mode. A missing source reader blocks submission unless the explicit grant flag is used successfully. Current external-asset disposition is explicit: | Dependency | Current action | Reference status | | --- | --- | --- | | URI file/folder and MLTable bytes | `copy` or `reference` | Dataset V3 references are implemented and live-proven. | | OCI environment image | `reference` | Reused in place through `environmentImageReference`; digest pinning is recommended. | | Code snapshot | `copy` | Dataset V3 code references are a feasible follow-up, but not implemented/live-proven. | | AML model | `copy` | Re-registered through Model V3; an external-storage reference contract is not yet proven. | | Outputs | `allocate` | They are new results, so Foundry must write/register them rather than reference source outputs. | This means `reference-only` intentionally fails a normal code-and-model job today and names `asset.code` plus each model input in `nonReferenceableDependencyIds`. It is a roadmap signal, not a false claim that all bytes can already remain in place. ### Capability test contract Every analyzer finding is assigned to one of the catalog families in `CAPABILITY_FAMILY_CATALOG`. The analyzer fails closed if code attempts to emit a finding whose family is not cataloged. Every report embeds the full catalog and adds `family` plus `testCoverage` to each finding. The catalog ownership check is: ```powershell python -m pytest ` tests/unit/test_aml_command_job_analysis.py::test_capability_catalog_file_references_exist ` -q ``` That check fails when any file-like unit or live-evidence reference in the catalog does not exist. The rest of `tests/unit` exercises analyzer policy, translation, migration, equivalence, permissions, and release-validation behavior. RBAC has a dedicated matrix in `tests/unit/test_aml_command_job_permissions.py`. It covers inherited scope, read-versus-write data roles, generic Contributor rejection, ACR/Foundry role sets, conditional assignments, custom-role uncertainty, pagination, identity resolution, cross-subscription grouping, connection auth metadata, and the distinction between missing and uninspectable assignments. `unitTestCoverageComplete` in the report is therefore a guarded contract rather than an informal claim. Live coverage remains deliberately separate: `verification` describes the encountered variant, `liveEvidence` names existing tests/scenarios for its family, and `familiesWithoutLiveEvidence` exposes gaps. An unsupported or manual-action capability can be fully unit-tested while still having no live migration path; the report does not relabel that as supported. ```powershell aml-foundry-migrate migrate ` --source-subscription ` --source-resource-group ` --source-workspace ` --source-export-compute ` --source-job ` --project-endpoint https://.services.ai.azure.com ` --project-name ` --storage-connection ` --foundry-compute-id ` --foundry-instance-type ``` To avoid copying data inputs, register zero-copy Dataset V3 references instead: ```powershell aml-foundry-migrate migrate ` --source-subscription ` --source-resource-group ` --source-workspace ` --source-export-compute ` --source-job ` --dataset-transfer-mode reference ` --source-storage-connection ` --project-endpoint https://.services.ai.azure.com ` --project-name ` --storage-connection ` --foundry-compute-id ` --foundry-instance-type ``` `reference` applies to `uri_file`, `uri_folder`, and MLTable-backed data inputs. It resolves AML datastore paths to their HTTPS storage URIs and calls Dataset V3 `create_or_update` with `is_reference=true`; it does not run the AML data export job and does not upload those bytes to Foundry project storage. The Foundry connection must point to the source storage account, and the target job identity must have data-plane read access. Every reference migration reads the connection metadata and verifies that its target host matches every resolved data-input host before creating Foundry assets. Code snapshots and models are still uploaded because they use the distinct Dataset V3 code and Model V3 contracts. Output assets are still written and registered by the Foundry job. For URI-folder and MLTable inputs, the registered Dataset V3 URI is the AML container root and `migration-manifest.json` records `registeredDataUri` plus `foundryInputPathSuffix`; concatenating them must reproduce `referenceDataUri`. Use `--environment-image` when the AML environment was built from a Docker context or has a Conda overlay. Foundry accepts a runnable image reference; it does not rebuild the AML environment definition. Use `--source-code-path` when the source job does not expose a versioned workspace code asset, such as a registry-backed or externally managed code snapshot. ## Primitive changes | AML command-job primitive | Foundry representation | Migration behavior | | --- | --- | --- | | `code` workspace asset | Dataset V3 `codeId` | Downloads the immutable AML code version and uploads the folder. | | AML environment asset | `environmentImageReference` | Resolves an image-only environment; rejects Conda/build definitions unless an image override is supplied. | | AML compute name | Cognitive Services `computeId` plus `resources.instanceType` | Replaced by the target settings; AML VM size is not copied. | | `string`, `integer`, `number`, `boolean` inputs | `jobInputType: literal` | Converts the value to the Foundry literal wire shape. | | `uri_file`, `uri_folder`, `mltable` inputs | Dataset V3 `azureai://.../data/.../versions/...` | Defaults to exporting bytes through source compute and uploading them to project storage. Optional `reference` mode registers the existing source storage URI without copying bytes. | | custom/MLflow/SafeTensors/Triton model inputs | Model V3 `azureai://.../models/.../versions/...` | Uses AML model download first; falls back to source-compute export, then uploads customer bytes under `model/`, registers the model, and targets that payload path from command placeholders. This keeps Model V3's `manifest.base.json` outside the path seen by the migrated command. | | AML datastore output path | Foundry output `assetName` plus `assetVersion` | Drops the workspace datastore path and allocates a project-scoped output asset. AML MLTable outputs are registered as URI folders while retaining the `MLTable` file and payload. | | input/output delivery modes | Foundry delivery modes | Maps input modes to their Pascal-cased equivalents. AML `upload` outputs use Foundry `ReadWriteMount`, the service's completed-job registration path for `uri_file`, `uri_folder`, and model outputs. | | templated environment variable | shell assignment in `properties.command` | Foundry expands input/output placeholders in the command, not in `environmentVariables`. Static variables remain unchanged. | | `limits.timeout` seconds | ISO-8601 `Command` limits | For example, `3600` becomes `PT3600S`. | | AML distribution | Foundry distribution discriminator | Supports MPI, PyTorch, TensorFlow, and Ray fields. | | SLA tier and queue priority | `resources.properties.AISuperComputer.slaTier` plus source queue fields | The CLI defaults the target SLA to Premium. `--target-job-tier Standard` is available when the target account has Standard quota; `Preserve` omits the target override. Source queue tier/priority remain separate translated fields when the source API exposes them. | | user identity | `userAssignedIdentityId` | Uses the explicitly supplied target UAI. AML identity objects are not copied across resource boundaries. | | generated Studio/Tracking service | Foundry-generated links | Omits AML-generated endpoints. Explicit SSH/Jupyter/TensorBoard/VS Code/Theia/Grafana/custom service definitions are translated. | Descriptions, experiment name, tags, instance count, shared-memory size, named inputs, named outputs, output descriptions/tags, distribution settings, timeout, queue tier, priority, and supported interactive services are retained. ## Compatibility warnings Before transferring assets, the migrator audits the materialized AML command job and prints `WARNING:` records for source semantics that cannot be preserved. The same warnings are retained in `migration-manifest.json` and returned by the CLI. The audit currently covers: - source identity replacement, parent-job lineage, notification settings, and non-default deterministic/caching semantics - source instance type replacement and AML-only Docker/location settings - fixed `path_on_compute`, input defaults/optionality/ranges/enums, explicit datastore selection, intellectual-property metadata, and early outputs - alternate model formats that are translated but do not yet have live migration parity coverage; AML MLTable outputs use the live-validated URI-folder adaptation described above - AML runtime-only properties, which are filtered while portable custom and MLflow lineage properties are retained - mutable image tags and private-ACR identity/network prerequisites Unsupported input/output types, modes, distribution types, priorities, service types, or unresolved environments remain hard errors rather than warnings. This prevents the tool from submitting a job whose behavior it already knows it cannot represent. ## Environment migration Foundry command jobs consume `environmentImageReference`, so image-backed AML environments are reused **in place**. The migrator resolves a versioned AML environment asset to its image reference and submits that OCI reference; it does not download image layers or push them to another registry. This works for MCR, public registries, and direct ACR references that Foundry compute can reach. For private ACR images, the target UAI needs `AcrPull`, and Foundry compute must have working DNS and network reachability to the registry and its data endpoint. AML workspace identity, registry credentials, private endpoints, and firewall rules are not copied. Tag-based references are accepted but produce a warning; pin `repository@sha256:` when reproducibility matters. Environment behavior beyond the image itself: - static environment variables are copied to the Foundry job - input/output-templated variables are moved into the shell command because Foundry expands placeholders there, not in `environmentVariables` - AML Conda overlays and Docker build contexts are rejected; publish an equivalent image first and pass `--environment-image` - AML environment tags, descriptive properties, build datastore, and asset version metadata are not runtime dependencies and are not recreated - source compute image caches, Docker arguments, node locations, secrets, certificates, proxy configuration, and managed-network rules are not inferred - GPU/CUDA/driver compatibility, CPU architecture, entrypoint, working directory, user, and native libraries remain properties of the referenced image and the chosen Foundry compute SKU Copying an image to a target ACR is not currently part of the migration CLI. That can be added as a separate `reference` versus `import` policy, but an import must also handle cross-tenant authorization, multi-architecture manifests, signatures/SBOMs, immutable digest mapping, private endpoints, and target-UAI `AcrPull`. Reusing a digest-pinned source image is the lower-risk default. ## Coverage tiers The `exercise` and `qualify-release` commands are operator-invoked live workflows, not pytest tests. Their rich fixture covers a single-node command job with code, primitive and asset inputs, static and templated environment variables, timeout/shared memory, and data/model outputs. A retained report counts as live evidence only after the workflow completes against configured AML and Foundry resources; command availability or offline collection is not evidence. No component-by-component live matrix ships with this sample. The following are translated and unit-tested but still need migration-specific live parity tests: multi-node MPI, PyTorch/TensorFlow/Ray distributions, mount/direct/evaluation delivery modes, Jupyter/Theia/Grafana/custom services, alternate queue/priority values, and alternate MLflow/SafeTensors/Triton model formats. Ray's current SDK fields (`port`, `address`, dashboard settings, and head/worker arguments) are preserved. The following require an explicit policy rather than transparent translation: nontrivial MLTable transformations, optional or constrained component inputs, early-available outputs, source output destinations, job lineage/notifications, secret-backed settings, and registry-backed environments or code assets. ## Asset-transfer behavior Data is never copied by parsing datastore URLs or exporting credentials. The script submits an AML staging command that binds each source URI using normal AML input resolution, copies the mounted/downloaded content to named outputs, and downloads those outputs through the AML SDK. This supports workspace data assets, datastore URIs, registry or cross-workspace references accepted by the source job, and external URIs that the source compute can read. Source workspaces may disable storage-account shared keys. The exercise handles that configuration without changing the account policy: fixture bytes and staging outputs use `AzureCliCredential` against `workspaceblobstore`, and source jobs run under AML user identity. Code uses AML's `startPendingUpload` handshake, uploads to the service-assigned temporary container with Entra ID, stamps the indicator blob with `upload_status=completed` plus asset name/version, and registers the local content hash. This produces an AML-protected snapshot; merely registering an arbitrary HTTPS blob folder is rejected by AML before user code starts. The formal live fixture uses download/upload delivery for source data and outputs. On some classic AML compute stacks, `ro_mount`/`rw_mount` still enters `UriMountSession` and fails with `ScriptExecution.StreamAccess.Authentication` when shared keys are disabled, even when the submitting user has Blob Data Contributor. Mount-mode translation remains covered by unit contract tests; use download/upload for the live source oracle unless that AML workspace's mount identity path has been configured and verified separately. Foundry uploads use the target project's named storage connection. Temporary SAS credentials returned by Dataset V3 or Model V3 remain in memory and are never written to the manifest or console. Model V3 adds service metadata at the asset root, so migrated model payloads are deliberately stored under `model/` and model placeholders in the command are translated to that subdirectory. ## Resume and audit files Each run writes under `--work-dir`: - `migration-manifest.json`: source/target identity, sanitized source job, downloaded paths, uploaded asset IDs, export job, target job, warnings, status, and integrity fingerprints - `foundry-job-request-attempt-N.json`: the immutable translated Foundry request for attempt `N`, with secret-like values and signed query parameters redacted - `source-code/`: downloaded AML code snapshot - `inputs/`: downloaded registered models - `export-code/` and `export-download/`: batched AML dependency export - `validation/`: downloaded Foundry outputs for an `exercise` run - `evidence/aml/`: AML stream log and keyless source-output downloads - `evidence/foundry/`: Foundry `user_logs/*` and normalized output references - `equivalence-report.json`: formal live-test result and comparison digests When `--work-dir` is omitted, runs are stored under `~/.aml-foundry-migration/runs/`. Set `AML_MIGRATION_RUNS_DIR` to use a different base directory. Migration manifests and saved request JSON are sanitized before writing, but downloaded code, data, models, outputs, and raw service logs can contain customer content. Treat the complete work directory as sensitive evidence and do not place it in source control. The manifest is updated atomically after each expensive operation. Rerunning the same command with the same work directory reuses completed downloads, uploads, the AML export job, and the submitted Foundry job. Resume is bound to fingerprints of the complete correctness-relevant invocation, the materialized AML source-job definition, each source URI, and the submitted Foundry request. Any mismatch is rejected before stale assets or a stale job can be reused. Polling interval, timeout, and wait/no-wait remain adjustable operational controls. Manifests from older builds that lack complete resume fingerprints require a new work directory. When waiting for completion, only Foundry `Completed` is successful. `Failed`, `Canceled`, or `Cancelled` terminates the migration with a nonzero CLI exit code while retaining the recorded status for diagnosis. `Paused` remains pollable because live jobs can resume through `Paused` to `Running` and `Completed`. Choose a new work directory to intentionally produce a second independent copy. Output asset names include a run suffix because Foundry output versions are immutable and reuse can return a conflict. ## Requirements and permissions - Python 3.10 or newer with this package's dependencies installed - Azure CLI authenticated to the source and target tenant - AML workspace read/job-create permissions - permission to use the selected AML compute and workspace storage - permission to download AML code and model assets - `Storage Blob Data Contributor` (or equivalent data-plane rights) on the AML workspace storage account when shared-key access is disabled - Foundry project access, including Dataset V3 and Model V3 create/read - access to the project storage connection - permission to submit to the selected Foundry compute - target UAI RBAC on private images or data when a UAI is supplied - `Microsoft.Authorization/roleAssignments/write` on resolved AML source storage accounts only when `--grant-reference-storage-access` is requested The workflow leaves source assets, source/export jobs, Foundry assets, and the Foundry job in place for inspection. Delete them with the owning service's normal retention or cleanup tooling after validation. ## Deliberate boundaries - No pipelines, components, sweeps, AutoML, or pipeline-child reconstruction. - AML Conda overlays and Docker build contexts require a prebuilt image override. - AML compute-specific Docker arguments and location hints are reported but not copied because Foundry compute owns those settings. - A non-versioned code reference requires `--source-code-path`. - Secret values are not migrated. Recreate secret-backed settings as Foundry connections or target identity access rather than writing secrets into a job. - Network-isolated source data must be readable by the selected AML export compute; network-isolated target assets must be readable by Foundry compute. --- ### Sdk Python Foundation Models System Docs Component Docs Image Finetune Mmd Finetune Component (`sdk/python/foundation-models/system/docs/component_docs/image_finetune/mmd_finetune_component.md`) # MMDetection Model Finetune Component This component enables finetuning of pretrained models on custom or pre-available datasets. The component supports Deepspeed and ONNXRuntime configurations for performance enhancement. The components can be seen in your workspace component page as below: - [mmdetection_image_objectdetection_instancesegmentation_model_import](https://ml.azure.com/registries/azureml/components/mmdetection_image_objectdetection_instancesegmentation_model_import) - [mmdetection_image_objectdetection_instancesegmentation_finetune](https://ml.azure.com/registries/azureml/components/mmdetection_image_objectdetection_instancesegmentation_finetune) - [mmdetection_image_objectdetection_instancesegmentation_pipeline](https://ml.azure.com/registries/azureml/components/mmdetection_image_objectdetection_instancesegmentation_pipeline) # 1. Inputs 1. _model_path_ (URI_FOLDER, required) Path to the output directory of [model import component](mmd_model_import_component.md/#2-outputs). 2. _training_data_ (MLTABLE, required) Path to the mltable folder of training dataset. 3. _validation_data_ (MLTABLE, optional) Path to the mltable folder of validation dataset. 4. _image_min_size_ (int, optional) Minimum image size after augmentation that is input to the network. If left empty, it would either be overwritten by image_scale in model config or would be chosen based on the task type and model selected. The image will be rescaled as large as possible within the range [image_min_size, image_max_size]. The image size will be constraint so that the max edge is no longer than image_max_size and short edge is no longer than image_min_size. 5. _image_max_size_ (int, optional) Maximum image size after augmentation that is input to the network. If left empty, it would either be overwritten by image_scale in model config or would be chosen based on the task type and model selected. The image will be rescaled as large as possible within the range [image_min_size, image_max_size]. The image size will be constraint so that the max edge is no longer than image_max_size and short edge is no longer than image_min_size. 6. _task_name_ (string, required) Which task the model is solving. It could be one of [`image-object-detection`, `image-instance-segmentation`]. 7. _metric_for_best_model_ (string, optional) Specify the metric to use to compare two different models. It could be one of [`mean_average_precision`, `precision`, `recall`]. If left empty, will be chosen automatically based on the task type and model selected. Generally, `mean_average_precision` is chosen for object detection and instance segmentation tasks. 8. _apply_augmentations_ (bool, optional) If set to true, will enable data augmentations for training and validation. Please note, if it is set to false, normalization and resize augmentation would still be applied to pre-process the image. The default value is true. 9. _number_of_workers_ (int, optional) Number of subprocesses to use for data loading (PyTorch only). 0 means that the data will be loaded in the main process. The default value is 8. 10. _apply_deepspeed_ (bool, optional) If true enables deepspeed. If no `deepspeed_config` is provided, the default config in the component will be used else the user passed config will be used. The default value is false. Please note that to enable deepspeed, `apply_deepspeed` must be set to true. Please note deepspeed is not yet supported for MMDetection, will be enabled in future. 11. _deepspeed_config_ (URI_FILE, optional) Path to the deepspeed config file. Please note deepspeed is not yet supported for MMDetection models, will be enabled in future. 12. _apply_ort_ (bool, optional) If true, apply ORT optimization. The default value is false. Please note ORT is not yet supported for MMDetection models, will be enabled in future. 13. _number_of_epochs_ (int, optional) Number of training epochs. If left empty, will be chosen automatically based on the task type and model selected. 14. _max_steps_ (int, optional) If set to a positive number, the total number of training steps to perform. Overrides 'number_of_epochs'. In case of using a finite iterable dataset the training may stop before reaching the set number of steps when all data is exhausted. If left empty, will be chosen automatically based on the task type and model selected. 15. _training_batch_size_ (int, optional) Batch size used for training. If left empty, will be chosen automatically based on the task type and model selected. 16. _validation_batch_size_ (int, optional) Batch size used for validation. If left empty, will be chosen automatically based on the task type and model selected. 17. _auto_find_batch_size_ (bool, optional) If set to true, the train batch size will be automatically downscaled recursively till if finds a valid batch size that fits into memory. If the provided 'per_device_train_batch_size' goes into Out Of Memory (OOM) enabling auto_find_batch_size will find the correct batch size by iteratively reducing 'per_device_train_batch_size' by a factor of 2 till the OOM is fixed. The default value is false. 18. _learning_rate_ (float, optional) Start learning rate used for training. If left empty, will be chosen automatically based on the task type and model selected. 19. _learning_rate_scheduler_ (string, optional) The learning rate scheduler to use. The default value is warmup_linear. It could be one of [`warmup_linear`, `warmup_cosine`, `warmup_cosine_with_restarts`, `warmup_polynomial`, `constant`, `warmup_constant`]. If left empty, will be chosen automatically based on the task type and model selected. 20. _warmup_steps_ (int, optional) The number of steps used for the learning rate scheduler warmup phase. It is the number of steps used for a linear warmup from 0 to learning_rate. If left empty, will be chosen automatically based on the task type and model selected. 21. _optimizer_ (string, optional) Optimizer to be used while training. The default value is adamw_hf. It could be one of [`adamw_hf`, `adamw`, `sgd`, `adafactor`, `adagrad`]. If left empty, will be chosen automatically based on the task type and model selected. 22. _weight_decay_ (float, optional) The weight decay to apply (if not zero) to all layers except all bias and LayerNorm weights in AdamW optimizer. If left empty, will be chosen automatically based on the task type and model selected. 23. _extra_optim_args_: (string, optional) Optional additional arguments that are supplied to SGD Optimizer. The arguments should be semi-colon separated key value pairs and should be enclosed in double quotes. For example, "momentum=0.5; nesterov=True" for sgd. Please make sure to use a valid parameter names for the chosen optimizer. For exact parameter names, please refer to https://pytorch.org/docs/1.13/generated/torch.optim.SGD.html#torch.optim.SGD for SGD. Parameters supplied in extra_optim_args will take precedence over the parameter supplied via other arguments such as weight_decay. If weight_decay is provided via "weight_decay" parameter and via extra_optim_args both, values specified in extra_optim_args will be used. 24. _gradient_accumulation_step_ (int, optional) Number of updates steps to accumulate the gradients for, before performing a backward/update pass. If left empty, will be chosen automatically based on the task type and model selected. 25. _precision_ (int, optional) Apply mixed precision training. This can reduce memory footprint by performing operations in half-precision. It could one of [`16`, `32`]. The default value is "32". 26. _iou_threshold_ (float, optional) This is the IOU threshold used during inference for non-maximum suppression while post processing the predictions. If left empty, will be chosen automatically based on the task type and model selected. 27. _box_score_threshold_ (float, optional) During inference, only return proposals with a score greater than `box_score_threshold`. The score is the multiplication of the objectness score and classification probability. If left empty, will be chosen automatically based on the task type and model selected. 28. _random_seed_ (int, optional) Random seed that will be set at the beginning of training. The default value is 42. 29. _evaluation_strategy_ (string, optional) The evaluation strategy to adopt during training. If set to "steps", either the `evaluation_steps_interval` or `evaluation_steps` needs to be specified, which helps to determine the step at which the model evaluation needs to be computed else evaluation happens at end of each epoch. The default value is "epoch". It could be one of [`epoch`, `steps`]. 30. _evaluation_steps_ (int, optional) Number of update steps between two evals if evaluation_strategy='steps'. The default value is 500. 31. _logging_strategy_ (string, optional) The logging strategy to adopt during training. If set to "steps", the `logging_steps` will decide the frequency of logging else logging happens at the end of epoch. The default value is "epoch". It could be one of [`epoch`, `steps`]. 32. _logging_steps_ (int, optional) Number of update steps between two logs if logging_strategy='steps'. The default value is 500. 33. _save_strategy_ (string, optional) The checkpoint save strategy to adopt during training. The default value is "epoch". It could be one of [`epoch`, `steps`]. Please note that the save_strategy and evaluation_strategy should match. 34. _save_steps_ (int, optional) Number of updates steps before two checkpoint saves if save_strategy="steps". The default value is 500. Please note that the saving steps should be a multiple of the evaluation steps. 35. _save_total_limit_ (int, optional) If a value is passed, will limit the total number of checkpoints. Deletes the older checkpoints in output_dir. If the value is -1 saves all checkpoints". The default value is -1. 36. _early_stopping_ (bool, optional) If set to true, early stopping is enabled. The default value is false. 37. _early_stopping_patience_ (int, optional) Stop training when the specified metric doesn't improve for early_stopping_patience evaluation calls. The default value is 1. 38. _max_grad_norm_ (float, optional) Maximum gradient norm (for gradient clipping). If left empty, will be chosen automatically based on the task type and model selected. 39. _resume_from_checkpoint_ (bool, optional) If set to true, resumes the training from last saved checkpoint. Along with loading the saved weights, saved optimizer, scheduler and random states will be loaded if exists. The default value is false. 40. _save_as_mlflow_model_ (bool, optional) Save as mlflow model with pyfunc as flavour. The default value is true. # 2. Outputs 1. _output_dir_pytorch_ (custom_model, required) The folder containing finetuned model output with checkpoints, model config, optimizer and scheduler states and random number states in case of distributed training. 2. _output_dir_mlflow_ (URI_FOLDER, optional) Output directory to save the finetuned model as mlflow model. # 4. Run Settings This setting helps to choose the compute for running the component code. **For the purpose of finetune, gpu compute should be used**. We recommend using Standard_NC6s_v3 compute. > Select *Use other compute target* - Under this option, you can select either `compute_cluster` or `compute_instance` as the compute type and the corresponding instance / cluster created in your workspace. - If you have not created the compute, you can create the compute by clicking the `Create Azure ML compute cluster` link that's available while selecting the compute. See the figure below ## 4.1. Settings for Distributed Training > When creating the compute, set the `Maximum number of nodes` to the desired value for multi-node training as shown in the figure below > In case of distributed training, also known as multi-node training, the mode must be set to `Mount` ( not `Upload`) as shown in the figure below > Set the number of processes under Distribution subsection to use all the gpus in a node To use all the gpus within a node, set the `Process count per instance` to number of gpus in that node as shown below > Set the number of nodes under the Resources subsection In case of distributed training, you can configure `instance count` under this subsection to increase the number of nodes as shown below --- ### Sdk Python Foundation Models System Docs Component Docs Image Finetune Mmd Model Import Component (`sdk/python/foundation-models/system/docs/component_docs/image_finetune/mmd_model_import_component.md`) # MMDetection Model Import Component The component copies the input model folder to the component output directory when the model is passed as an input to the `pytorch_model` or `mlflow_model` nodes. If `model_name `is selected, the component will download the config for the model from [MMDetection model zoo](https://github.com/open-mmlab/mmdetection/blob/v3.1.0/docs/en/model_zoo.md). The component can be seen in your workspace component page - [mmdetection_image_objectdetection_instancesegmentation_model_import](https://ml.azure.com/registries/azureml/components/mmdetection_image_objectdetection_instancesegmentation_model_import). # 1. Inputs 1. _pytorch_model_ (custom_model, optional) Pytorch Model registered in AzureML Asset. The continual finetune flag will be set to true in this case. If you want to resume from previous training state, set *resume_from_checkpoint* flag to True in [finetune component](mmd_finetune_component.md/#39-resume-from-checkpoint). 2. _mlflow_model_ (mlflow_model, optional) MlFlow Model registered in AzureML Asset. Some MMDetection models are registered in azureml registry and can be used directly. The user can also register MMDetection models into their workspace or organisation's registry, and use them. Following models are registered in azureml registry, and can be used directly. | Model Name | Source | | :------------: | :-------: | | [deformable-detr_refine_twostage_r50_16xb2-50e_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-deformable-detr_refine_twostage_r50_16xb2-50e_coco/version/8) | azureml registry | | [sparse-rcnn_r50_fpn_300-proposals_crop-ms-480-800-3x_coco-800_3x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-sparse-rcnn_r50_fpn_300-proposals_crop-ms-480-800-3x_coco-800_3x_coco/version/8) | azureml registry | | [sparse-rcnn_r101_fpn_300-proposals_crop-ms-480-800-3x_coco-800_3x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-sparse-rcnn_r101_fpn_300-proposals_crop-ms-480-800-3x_coco-800_3x_coco/version/8) | azureml registry | | [vfnet_r50-mdconv-c3-c5_fpn_ms-2x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-vfnet_r50-mdconv-c3-c5_fpn_ms-2x_coco/version/8) | azureml registry | | [vfnet_x101-64x4d-mdconv-c3-c5_fpn_ms-2x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-vfnet_x101-64x4d-mdconv-c3-c5_fpn_ms-2x_coco/version/8) | azureml registry | | [yolof_r50_c5_8x8_1x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-yolof_r50_c5_8x8_1x_coco/version/8) | azureml registry | | [mmd-3x-mask-rcnn_swin-t-p4-w7_fpn_1x_coco](https://ml.azure.com/registries/azureml/models/mmd-3x-mask-rcnn_swin-t-p4-w7_fpn_1x_coco/version/8) | azureml registry | Below is the folder structure of a registered MLFlow model. - In the finetuned model, all the augmentations should be specified in artifacts/augmentations.yaml file. - All the model files should be stored in artifacts/*.pth or artifacts/*.py file. - **`MLmodel`** is a yaml file and this should contain relavant information. See the sample MLmodel file [here](../../sample_files/MMDMLmodel). Please note that the is kept as a placeholder only and in the your MLmodel file, you should see an absolute path to the artifact. > Currently _resume_from_checkpoint_ is **NOT** fully enabled with _mlflow_model_. Only the saved model weights can be reloaded but not the optimizer, scheduler and random states **NOTE** The _pytorch_model_ take priority over _mlflow_model_, in case both inputs are passed # 2. Outputs 1. _output_dir_ (URI_FOLDER): Path to output directory which contains the component metadata and the copied model data. # 3. Parameters 1. _model_family_ (string, required) Which framework the model belongs to. It could be one of [`MmDetectionImage`] 2. _model_name_ (string, optional) Please select models from AzureML Model Assets for all supported models. For MMDetection models, which are not supported in AzureML model registry, the model's config name is required, same as it's specified in MMDetection Model Zoo. For e.g. fast_rcnn_r101_fpn_1x_coco for [this config file](https://github.com/open-mmlab/mmdetection/blob/master/configs/fast_rcnn/fast_rcnn_r101_fpn_1x_coco.py). You can see the comprehensive list of model configs [here](https://github.com/open-mmlab/mmdetection/tree/v3.1.0/configs) and the documentation of model zoo [here](https://github.com/open-mmlab/mmdetection/blob/v3.1.0/docs/en/model_zoo.md). Please note that it is the user responsibility to comply with the model's license terms. 3. __download_from_source__ (boolean, optional) Set to True for downloading model directly from MMDetection instead of system registry. Defaults to `False`. # 4. Run Settings This setting helps to choose the compute for running the component code. For the purpose of model selector, cpu compute should work. We recommend using D12 compute. 1. Option1: *Use default compute target* If this option is selected, it will identify the compute from setting tab on top right as shown in the below figure 2. Option2: *Use other compute target* - Under this option, you can select either `compute_cluster` or `compute_instance` as the compute type and select any of the already created compute in your workspace. - If you have not created the compute, you can create the compute by clicking the `Create Azure ML compute cluster` link that's available while selecting the compute. See the figure below. --- ### Sdk Python Foundation Models System Docs Component Docs Image Finetune Transformers Finetune Component (`sdk/python/foundation-models/system/docs/component_docs/image_finetune/transformers_finetune_component.md`) # Transformers Model Finetune Component This component enables finetuning of pretrained models on custom or pre-available datasets. The component supports Deepspeed and ONNXRuntime configurations for performance enhancement. The components can be seen in your workspace component page as below: - [transformers_image_classification_model_import](https://ml.azure.com/registries/azureml/components/transformers_image_classification_model_import) - [transformers_image_classification_finetune](https://ml.azure.com/registries/azureml/components/transformers_image_classification_finetune) - [transformers_image_classification_pipeline](https://ml.azure.com/registries/azureml/components/transformers_image_classification_pipeline) # 1. Inputs 1. _model_path_ (URI_FOLDER, required) Path to the output directory of [model import component](transformers_model_import_component.md/#2-outputs). 2. _training_data_ (MLTABLE, required) Path to the mltable folder of training dataset. 3. _validation_data_ (MLTABLE, optional) Path to the mltable folder of validation dataset. 4. _image_height_ (int, optional) Final Image height after augmentation that is input to the network. Default value is -1 which means it would be overwritten by default image height in Hugging Face feature extractor. If either image_width or image_height is set to -1, default value would be used for both width and height. 5. _image_width_ (int, optional) Final Image width after augmentation that is input to the network. Default value is -1 which means it would be overwritten by default image width in Hugging Face feature extractor. If either image_width or image_height is set to -1, default value would be used for both width and height. 6. _task_name_ (string, required) Which task the model is solving. It could be one of [`image-classification`, `image-classification-multilabel`]. 7. _metric_for_best_model_ (string, optional) Specify the metric to use to compare two different models. If left empty, will be chosen automatically based on the task type selected. It could be one of [`loss`, `f1_score_macro`, `accuracy`, `precision_score_macro`, `recall_score_macro`, `iou`, `iou_macro`, `iou_micro`, `iou_weighted`]. If selecting by yourself, use iou_* metrics in case of multi-label classification task. Generally, `accuracy` is chosen for multi-class classification task, and `iou` is chosen for multi-label classification task. 8. _apply_augmentations_ (bool, optional) If set to true, will enable data augmentations for training. The default value is true. 9. _number_of_workers_ (int, optional) Number of subprocesses to use for data loading (PyTorch only). 0 means that the data will be loaded in the main process. The default value is 8. 10. _apply_deepspeed_ (bool, optional) If true enables deepspeed. If no `deepspeed_config` is provided, the default config in the component will be used else the user passed config will be used. The default value is false. Please note that to enable deepspeed, `apply_deepspeed` must be set to true. 11. _deepspeed_config_ (URI_FILE, optional) Path to the deepspeed config file. 12. _apply_ort_ (bool, optional) If true apply ORT optimization. The default value is false. 13. _number_of_epochs_ (int, optional) Number of epochs to run for finetune. If left empty, will be chosen automatically based on the task type and model selected. 14. _max_steps_ (int, optional) If set to a positive number, it's the total number of training steps to perform. It overrides `number_of_epochs`. In case of using a finite iterable dataset the training may stop before reaching the set number of steps when all data is exhausted. If left empty, will be chosen automatically based on the task type and model selected. 15. _training_batch_size_ (int, optional) Batch size used for training. If left empty, will be chosen automatically based on the task type and model selected. 16. _validation_batch_size_ (int, optional) Batch size used for validation. If left empty, will be chosen automatically based on the task type and model selected. 17. _auto_find_batch_size_ (bool, optional) If set to true, the train batch size will be automatically downscaled recursively till if finds a valid batch size that fits into memory. If the provided 'per_device_train_batch_size' goes into Out Of Memory (OOM) enabling auto_find_batch_size will find the correct batch size by iteratively reducing 'per_device_train_batch_size' by a factor of 2 till the OOM is fixed. The default value is false. 18. _learning_rate_ (float, optional) Start learning rate used for training. If left empty, will be chosen automatically based on the task type and model selected. 19. _learning_rate_scheduler_ (string, optional) The learning rate scheduler to use. It could be one of [`warmup_linear`, `warmup_cosine`, `warmup_cosine_with_restarts`, `warmup_polynomial`, `constant`, `warmup_constant`]. If left empty, will be chosen automatically based on the task type and model selected. 20. _warmup_steps_ (int, optional) The number of steps for the learning rate scheduler warmup phase. If left empty, will be chosen automatically based on the task type and model selected. 21. _optimizer_ (string, optional) Optimizer to be used while training. 'adamw_ort_fused' optimizer is only supported for ORT training. It could be one of [`adamw_hf`, `adamw`, `sgd`, `adafactor`, `adagrad`, `adamw_ort_fused`] If left empty, will be chosen automatically based on the task type and model selected. 22. _weight_decay_ (float, optional) The weight decay to apply (if not zero) to all layers except all bias and LayerNorm weights in AdamW optimizer. If left empty, will be chosen automatically based on the task type and model selected. 23. _extra_optim_args_: (string, optional) Optional additional arguments that are supplied to SGD Optimizer. The arguments should be semi-colon separated key value pairs and should be enclosed in double quotes. For example, "momentum=0.5; nesterov=True" for sgd. Please make sure to use a valid parameter names for the chosen optimizer. For exact parameter names, please refer to https://pytorch.org/docs/1.13/generated/torch.optim.SGD.html#torch.optim.SGD for SGD. Parameters supplied in extra_optim_args will take precedence over the parameter supplied via other arguments such as weight_decay. If weight_decay is provided via "weight_decay" parameter and via extra_optim_args both, values specified in extra_optim_args will be used. 24. _gradient_accumulation_step_ (int, optional) Number of updates steps to accumulate the gradients for, before performing a backward/update pass. If left empty, will be chosen automatically based on the task type and model selected. 25. _precision_ (int, optional) Apply mixed precision training. This can reduce memory footprint by performing operations in half-precision. It could one of [`16`, `32`]. The default value is "32". 26. _label_smoothing_factor_ (float, optional) The label smoothing factor to use in range `[0.0, 1,0)`. Zero means no label smoothing, otherwise the underlying onehot-encoded labels are changed from 0s and 1s to label_smoothing_factor/num_labels and 1 - label_smoothing_factor + label_smoothing_factor/num_labels respectively. Not applicable to multi-label classification. If left empty, will be chosen automatically based on the task type and model selected. 27. _random_seed_ (int, optional) Random seed that will be set at the beginning of training. The default value is 42. 28. _evaluation_strategy_ (string, optional) The evaluation strategy to adopt during training. If set to "steps", either the `evaluation_steps_interval` or `evaluation_steps` needs to be specified, which helps to determine the step at which the model evaluation needs to be computed else evaluation happens at end of each epoch. The default value is "epoch". It could be one of [`epoch`, `steps`]. 29. _evaluation_steps_ (int, optional) Number of update steps between two evals if evaluation_strategy='steps'. The default value is 500. 30. _logging_strategy_ (string, optional) The logging strategy to adopt during training. If set to "steps", the `logging_steps` will decide the frequency of logging else logging happens at the end of epoch. The default value is "epoch". It could be one of [`epoch`, `steps`]. 31. _logging_steps_ (int, optional) Number of update steps between two logs if logging_strategy='steps'. The default value is 500. 32. _save_strategy_ (string, optional) The checkpoint save strategy to adopt during training. The default value is "epoch". It could be one of [`epoch`, `steps`]. 33. _save_steps_ (int, optional) Number of updates steps before two checkpoint saves if save_strategy="steps". The default value is 500. 34. _save_total_limit_ (int, optional) If a value is passed, will limit the total number of checkpoints. Deletes the older checkpoints in output_dir. If the value is -1 saves all checkpoints". The default value is -1. 35. _early_stopping_ (bool, optional) If set to true, early stopping is enabled. The default value is false. 36. _early_stopping_patience_ (int, optional) Stop training when the specified metric doesn't improve for early_stopping_patience evaluation calls. The default value is 1. 37. _max_grad_norm_ (float, optional) Maximum gradient norm (for gradient clipping). If left empty, will be chosen automatically based on the task type and model selected. 38. _resume_from_checkpoint_ (bool, optional) If set to true, resumes the training from last saved checkpoint. Along with loading the saved weights, saved optimizer, scheduler and random states will be loaded if exists. The default value is false. 39. _save_as_mlflow_model_ (bool, optional) Save as mlflow model with pyfunc as flavour. The default value is true. # 2. Outputs 1. _output_dir_pytorch_ (custom_model, required) The folder containing finetuned model output with checkpoints, model config, tokenizer, optimizer and scheduler states and random number states in case of distributed training. 2. _output_dir_mlflow_ (URI_FOLDER, optional) Output directory to save the finetuned model as mlflow model. # 4. Run Settings This setting helps to choose the compute for running the component code. **For the purpose of finetune, gpu compute should be used**. We recommend using Standard_NC6s or Standard_NC6s_v3 compute. > Select *Use other compute target* - Under this option, you can select either `compute_cluster` or `compute_instance` as the compute type and the corresponding instance / cluster created in your workspace. - If you have not created the compute, you can create the compute by clicking the `Create Azure ML compute cluster` link that's available while selecting the compute. See the figure below ## 4.1. Settings for Distributed Training > When creating the compute, set the `Maximum number of nodes` to the desired value for multi-node training as shown in the figure below > In case of distributed training, a.k.a multi-node training, the mode must be set to `Mount` (not `Upload`) as shown in the figure below > Set the number of processes under Distribution subsection to use all the gpus in a node To use all the gpus within a node, set the `Process count per instance` to number of gpus in that node as shown below > Set the number of nodes under the Resources subsection In case of distributed training, you can configure `instance count` under this subsection to increase the number of nodes as shown below --- ### Sdk Python Foundation Models System Docs Component Docs Image Finetune Transformers Model Import Component (`sdk/python/foundation-models/system/docs/component_docs/image_finetune/transformers_model_import_component.md`) # Transformers Model Import Component The component copies the input model folder to the component output directory when the model is passed as an input to the `pytorch_model` or `mlflow_model` nodes. If `model_name `is selected, the component is just a pass through. The component can be seen in your workspace component page - [transformers_image_classification_model_import](https://ml.azure.com/registries/azureml/components/transformers_image_classification_model_import). # 1. Inputs 1. _pytorch_model_ (custom_model, optional) Pytorch Model registered in AzureML Asset. The continual finetune flag will be set to true in this case. If you want to resume from previous training state, set *resume_from_checkpoint* flag to True in [finetune component](transformers_finetune_component.md/#38-resume-from-checkpoint). 2. _mlflow_model_ (mlflow_model, optional) MlFlow Model registered in AzureML Asset. Some HuggingFace models are registered in azureml registry and can be used directly. The user can also register HuggingFace models into their workspace or organisation's registry, and use them. Following models are registered in azureml registry, and can be used directly. | Model Name | Source | | ------ | ---------- | | [microsoft-beit-base-patch16-224-pt22k-ft22k](https://ml.azure.com/registries/azureml/models/microsoft-beit-base-patch16-224-pt22k-ft22k/version/11) | azureml registry | | [microsoft-swinv2-base-patch4-window12-192-22k](https://ml.azure.com/registries/azureml/models/microsoft-swinv2-base-patch4-window12-192-22k/version/11) | azureml registry | | [facebook-deit-base-patch16-224](https://ml.azure.com/registries/azureml/models/facebook-deit-base-patch16-224/version/10) | azureml registry | | [google-vit-base-patch16-224](https://ml.azure.com/registries/azureml/models/google-vit-base-patch16-224/version/10) | azureml registry | Here is the folder structure of a registered MLFlow model. - All the configuration files should be stored in _data/config_ - All the model files should be stored in _data/model_ - All the tokenizer files should be kept in _data/tokenizer_ - **`MLmodel`** is a yaml file and this should contain relavant information. See the sample MLmodel file [here](../../sample_files/HFMLmodel) > Currently _resume_from_checkpoint_ is **NOT** fully enabled with _mlflow_model_. Only the saved model weights can be reloaded but not the optimizer, scheduler and random states **NOTE** The _pytorch_model_ take priority over _mlflow_model_, in case both inputs are passed # 2. Outputs 1. _output_dir_ (URI_FOLDER): Path to output directory which contains the component metadata and the copied model data. # 3. Parameters 1. _model_family_ (string, required) Which framework the model belongs to. It could be one of [`HuggingFaceImage`] 2. _model_name_ (string, optional) Please select models from AzureML Model Assets for all supported models. For HuggingFace models, which are not supported in AzureML model registry, input HuggingFace model name here. You can see supported image-classification models [here](https://huggingface.co/models?pipeline_tag=image-classification&library=transformers). The model will be downloaded from HuggingFace hub using this model_name and is subject to third party license terms available on the HuggingFace model details page. It is the user's responsibility to comply with the model's license terms. 3. __download_from_source__ (boolean, optional) Set to True for downloading model directly from HuggingFace instead of system registry. Defaults to `False`. # 4. Run Settings This setting helps to choose the compute for running the component code. For the purpose of model selector, cpu compute should work. We recommend using D12 compute. 1. Option1: *Use default compute target* If this option is selected, it will identify the compute from setting tab on top right as shown in the below figure 2. Option2: *Use other compute target* - Under this option, you can select either `compute_cluster` or `compute_instance` as the compute type and select any of the already created compute in your workspace. - If you have not created the compute, you can create the compute by clicking the `Create Azure ML compute cluster` link that's available while selecting the compute. See the figure below. --- ### Sdk Python Foundation Models System Docs Component Docs Nlp Finetune Finetune Component Parameters (`sdk/python/foundation-models/system/docs/component_docs/nlp_finetune/finetune_component_parameters.md`) # Finetune Pipeline Component This component enables finetuning of pretrained models on custom datasets. The component supports Deepspeed for performance enhancement. The component supports following optimizations: 1. Parameter efficient finetuning with techniques like LoRA. 2. Supports Multi-GPU finetuning using Distributed Data Parallel (DDP) and DeepSpeed. 3. Supports Mixed Precision Training. 4. Supports Multi-Node training. 5. Supports flash attention for speed up finetuning and reducing memory footprint. At the time of writing, following tasks are supported through finetuning components: | Task | Notebook | | --- | --- | | Text Generation | https://github.com/Azure/azureml-examples/tree/main/sdk/python/foundation-models/system/finetune/Llama-notebooks/text-generation | Text Classification | https://github.com/Azure/azureml-examples/tree/main/sdk/python/foundation-models/system/finetune/text-classification | Named Entity Recognition/Token Classification | https://github.com/Azure/azureml-examples/tree/main/sdk/python/foundation-models/system/finetune/token-classification | Question Answering | https://github.com/Azure/azureml-examples/tree/main/sdk/python/foundation-models/system/finetune/question-answering | Summarization | https://github.com/Azure/azureml-examples/tree/main/sdk/python/foundation-models/system/finetune/summarization | Translation | https://github.com/Azure/azureml-examples/tree/main/sdk/python/foundation-models/system/finetune/translation Respective components can be found in the azureml registry: - [text_generation_pipeline](https://ml.azure.com/registries/azureml/components/text_generation_pipeline) - [text_classification_pipeline](https://ml.azure.com/registries/azureml/components/text_classification_pipeline) - [token_classification_pipeline](https://ml.azure.com/registries/azureml/components/token_classification_pipeline) - [question_answering_pipeline](https://ml.azure.com/registries/azureml/components/question_answering_pipeline) - [summarization_pipeline](https://ml.azure.com/registries/azureml/components/summarization_pipeline) - [translation_pipeline](https://ml.azure.com/registries/azureml/components/translation_pipeline) # 1. Inputs ## Model related inputs - _pytorch_model_path_ (custom_model, optional) Pytorch model asset path. Special characters like \ and ' are invalid in the parameter value. - _mlflow_model_path_ (mlflow_model, optional) Mlflow model asset path. Special characters like \ and ' are invalid in the parameter value **Note: one of the above two inputs is required.** ## Dataset related inputs - _train_file_path_ (uri_file, optional) Path to the registered training data asset. The supported data formats are `jsonl`, `json`, `csv`, `tsv` and `parquet`. Special characters like \ and ' are invalid in the parameter value. - _validation_file_path_ (uri_file, optional) Path to the registered validation data asset. The supported data formats are `jsonl`, `json`, `csv`, `tsv` and `parquet`. Special characters like \ and ' are invalid in the parameter value. - _test_file_path_ (uri_file, optional) Path to the registered test data asset. The supported data formats are `jsonl`, `json`, `csv`, `tsv` and `parquet`. Special characters like \ and ' are invalid in the parameter value. - _train_mltable_path_ (MLTABLE, optional) Path to the registered training data asset in `mltable` format. Special characters like \ and ' are invalid in the parameter value. - _validation_mltable_path_ (MLTABLE, optional) Path to the registered validation data asset in `mltable` format. Special characters like \ and ' are invalid in the parameter value. - _test_mltable_path_ (MLTABLE, optional) Path to the registered test data asset in `mltable` format. Special characters like \ and ' are invalid in the parameter value. ## Compute related inputs - _compute_model_import_ (string, optional, default: "serverless") compute to be used for model_import eg. provide 'FT-Cluster' if your compute is named 'FT-Cluster'. Special characters like \ and ' are invalid in the parameter value. If compute cluster name is provided, instance_type field will be ignored and the respective cluster will be used - _compute_preprocess_ (string, optional, default: "serverless") compute to be used for preprocess eg. provide 'FT-Cluster' if your compute is named 'FT-Cluster'. Special characters like \ and ' are invalid in the parameter value. If compute cluster name is provided, instance_type field will be ignored and the respective cluster will be used - _compute_finetune_ (string, optional, default: "serverless") compute to be used for finetune **NOTE: This has to be GPU compute** - _compute_model_evaluation_ (string, optional, default: "serverless") compute to be used for model_evaluation ### Serverless compute related parameters. Will be used only if the compute used is serverless - _instance_type_model_import_ (string, optional, default: "Standard_d12_v2") Instance type to be used for model_import component in case of serverless compute, eg. standard_d12_v2. The parameter compute_model_import must be set to 'serverless' for instance_type to be used - _instance_type_preprocess_ (string, optional, default: "Standard_d12_v2") Instance type to be used for preprocess component in case of serverless compute, eg. standard_d12_v2. The parameter compute_preprocess must be set to 'serverless' for instance_type to be used - _instance_type_finetune_ (string, optional, default: "Standard_nc24rs_v3") Instance type to be used for finetune component in case of serverless compute, eg. standard_nc6. The parameter compute_finetune must be set to 'serverless' for instance_type to be used - _instance_type_model_evaluation_ (string, optional, default: "Standard_nc24rs_v3") Instance type to be used for model_evaluation component in case of serverless compute, eg. standard_nc6. The parameter compute_model_evaluation must be set to 'serverless' for instance_type to be used ## MultiNode and MultiGPU training parameters - _number_of_gpu_to_use_finetuning_ (integer, optional, default: 1) number of gpus to be used per node for finetuning, should be equal to number of gpu per node in the compute SKU used for finetune. - _num_nodes_finetune_ (integer, optional, default: 1) number of nodes to be used for finetuning (used for distributed training). ## Data Preprocessing parameters - _batch_size_ (integer, optional, default: 1000) Number of examples to batch before calling the tokenization function - _max_seq_length_ (integer, optional, default: -1) - Default is -1 which means the padding is done up to the model's max length. Else will be padded to `max_seq_length`. - _pad_to_max_length_ (string, optional, default: "false") If set to True, the returned sequences will be padded according to the model's padding side and padding index, up to their `max_seq_length`. If no `max_seq_length` is specified, the padding is done up to the model's max length. ## Finetuning parameters ### LoRA parameters - _apply_lora_ (string, optional, default: false, allowed_values: [true, false]): Whether to enable LoRA for finetuning. If set to true, LoRA will be applied to the model. - _lora_alpha_ (integer, optional, default: 128) lora attention alpha - _lora_r_ (integer, optional, default: 8) lora dimension - lora_dropout (number, optional, default: 0.0) lora dropout value ### Deepspeed parameters - _apply_deepspeed_ (bool, optional, default: false) If true enables deepspeed. - _deepspeed_ (uri_file, optional, default: true) Deepspeed config to be used for finetuning. If no `deepspeed` is provided, the default config in the component will be used else the user passed config will be used. - _deepspeed_stage_ (string, optional, default: "2") Deepspeed stage to be used for finetuning. It could be one of [`2`, `3`]. Value '3' enabled model sharding across GPUs, useful if the model does not fit on a single GPU. ### Training parameters - _num_train_epochs_ (int, optional, default: 1) Number of epochs to run for finetune. - _max_steps_ (int, optional, default: -1) If set to a positive number, the total number of training steps to perform. Overrides 'epochs'. In case of using a finite iterable dataset the training may stop before reaching the set number of steps when all data is exhausted. - _per_device_train_batch_size_ (integer, optional, default: 1) Train batch size - _per_device_eval_batch_size_ (integer, optional, default: 1) Validation batch size - _auto_find_batch_size_ (bool, optional, default: false) If set to true, the train batch size will be automatically downscaled recursively till if finds a valid batch size that fits into memory. If the provided 'per_device_train_batch_size' goes into Out Of Memory (OOM) enabling auto_find_batch_size will find the correct batch size by iteratively reducing 'per_device_train_batch_size' by a factor of 2 till the OOM is fixed. - _learning_rate_ (number, optional, default: 0.00002) Start learning rate used for training. Defaults to linear scheduler. - _lr_scheduler_type_ (string, optional, default: linear) The learning rate scheduler to use. It could be one of [`linear`, `cosine`, `cosine_with_restarts`, `polynomial`, `constant`, `constant_with_warmup`]. If left empty, will be chosen automatically based on the task type and model selected. - _warmup_steps_ (integer, optional, default: 0) Number of steps used for a linear warmup from 0 to learning_rate. - _optim_ (string, optional, default: adamw_hf) Optimizer to be used while training. It could be one of [`adamw_hf`, `adamw_torch`, `adafactor`] If left empty, will be chosen automatically based on the task type and model selected. - _weight_decay_ (number, optional) The weight decay to apply (if not zero) to all layers except all bias and LayerNorm weights in AdamW optimizer - _gradient_accumulation_steps_ (integer, optional, default: 1) Number of updates steps to accumulate the gradients for, before performing a backward/update pass - _precision_ (string, optional, default: "32") Apply mixed precision training. This can reduce memory footprint by performing operations in half-precision. It could one of [`16`, `32`]. - _seed_ (int, optional, default: 42) Random seed that will be set at the beginning of training. - _evaluation_strategy_ (string, optional, default: epoch) The evaluation strategy to adopt during training. It could be one of [`epoch`, `steps`]. - _eval_steps_ (int, optional, default: 500) Number of update steps between two evals if evaluation_strategy='steps' - _logging_strategy_ (string, optional, default: steps) The logging strategy to adopt during training. It could be one of [`epoch`, `steps`]. - _logging_steps_ (integer, optional, default: 10) Number of update steps between two logs if logging_strategy='steps' - _save_total_limit_ (integer, optional, default: -1) If a value is passed, will limit the total number of checkpoints. Deletes the older checkpoints in output_dir. If the value is -1 saves all checkpoints". - _apply_early_stopping_ (string, optional, default: "false") If set to true, early stopping is enabled. The default value is false. - _early_stopping_patience_ (int, optional, default: 1) Stop training when the specified metric worsens for early_stopping_patience evaluation calls. - _max_grad_norm_ (number, optional, default: 1.0) Maximum gradient norm (for gradient clipping). - _resume_from_checkpoint_ (string, optional, default: "false") If set to true, resumes the training from last saved checkpoint. Along with loading the saved weights, saved optimizer, scheduler and random states will be loaded if exists. # 2. Outputs - _pytorch_model_folder_ (uri_folder) output folder containing _best_ model as defined by _metric_for_best_model_. Along with the best model, output folder contains checkpoints saved after every evaluation which is defined by the _evaluation_strategy_. Each checkpoint contains the model weight(s), config, tokenizer, optimzer, scheduler and random number states. - _mlflow_model_folder_ (mlflow_model) output folder containing _best_ finetuned model in mlflow format. - _evaluation_result_ (uri_folder) Test Data Evaluation Results --- --- METRICS --- - Files Extracted: 8 - Estimated Token Budget: ~21702 tokens - Canonical Reference: https://codewiki.google/github.com/Azure/azureml-examples