Summary  
This chapter explains how to use an iterative loop construct to process each item in a collection, leveraging dynamic content expressions to configure and run activities (such as scripting and data transfer) per iteration by feeding the loop with the output of a prior lookup.  

General domain of usage  
Data integration pipelines

The **ForEach activity** in Azure Data Factory is a powerful tool that enables you to perform **iterative operations**, such as processing multiple files or iterating through a list of items. 


**ForEach Activity** in Azure Data Factory acts as a loop, iterating over a collection of items, such as file names or table names, and performing specified actions like transforming or copying data for each item.

Definition

The **Lookup activity is often used in combination with the ForEach activity** to retrieve a list of items, such as file names or table names, for iteration. By passing the Lookup output to the ForEach activity, you can dynamically process each item in the collection, enabling scalable and flexible workflows in Azure Data Factory.

Download File

## How to Use ForEach Activity  

1. Start by using a **Lookup activity** to retrieve data;
2. Drag the **ForEach Activity** from the Activities pane onto the pipeline canvas;
3. In the **Settings** tab of the ForEach Activity, set the **Items** field to the output of the Lookup activity, such as:` @activity('LookupActivityName').output.value`;
4. Inside the ForEach Activity, add a **Copy Data Activity** or **any other activity** to process each item in the list dynamically;
5. In the **Source** tab of the Copy Data Activity, configure the input dataset for the data you want to process and use dynamic content to reference the current iteration. In our case we use the following query: `SELECT * FROM CovidData WHERE "HHSRegion" = '@{item().HHSRegion}'`;
6. In the **Sink** tab of the Copy Data Activity, configure the destination table name dynamically using the expression: ` TableName_@{item().ColumnName}`;
7. Validate and test the pipeline to ensure that data is correctly processed for each item in the dataset.

## @item() expression

In Azure Data Factory (ADF), the `@item()` expression is used inside the ForEach activity to **reference the current item** in the loop as it iterates through the collection of items. The `@item()` expression allows you to dynamically access data or values from the current iteration and use them in subsequent activities within the ForEach loop.

Its attributes refer to the **properties of the item being iterated over**. For example, if the loop is iterating through a list of objects, `@item()` can access specific attributes like `@item().Region` or `@item().FileName`, allowing dynamic referencing of data for each iteration.

In Azure Data Factory, you can use the `replace()` function to manipulate strings by replacing specific characters or substrings. For example, to replace all spaces (' ') with underscores ('_'), you would use the following expression:


    `replace('your string here', ' ', '_')`


    This function helps in cleaning and standardizing data, particularly when working with file names or table names that may contain spaces, making them easier to process programmatically.

Note

Which activity is commonly used in combination with the ForEach activity to retrieve a list of items for iteration?

Master the essentials of data engineering with Microsoft Azure in this comprehensive course. Starting with foundational concepts like cloud computing, resource management, and storage solutions, you'll progress to hands-on training in Azure Data Factory (ADF), mastering ETL/ELT workflows, and advanced data transformations. Through practical examples and real-world problem-solving, you'll gain the skills to design, implement, and optimize scalable data solutions using Azure.

Have a deep dive into the foundational concepts and tools required to begin your journey with Microsoft Azure. You'll explore cloud computing basics, learn how to organize and manage resources efficiently using Azure Resource Groups, and set up core services like Blob Storage and Azure SQL Database. Additionally, you’ll secure sensitive data with Azure Key Vault and gain an overview of Azure Data Factory (ADF) to understand its role in modern data engineering workflows.

Explore the core functionalities of Azure Data Factory (ADF), a powerful tool for creating and automating ETL/ELT workflows. In this section, you'll learn how to configure services, datasets, and pipelines while exploring key activities such as Copy, Lookup, and Script. You'll also work with advanced features like the ForEach activity for iteration and pipeline parameters for flexibility. 

This section focuses on leveraging Azure Data Factory's data flow capabilities to transform and enrich your data. You'll learn to connect various data sources, create derived columns, and merge datasets using joins. Practical examples will guide you through filtering, sorting, conditional splits, and aggregating data to generate meaningful insights. By mastering these transformations, you'll be equipped to build robust, efficient ETL workflows tailored to real-world data engineering challenges.

Apply your knowledge to real-world data engineering scenarios in this hands-on section. You'll tackle practical challenges like loading and analyzing data for different categories, identifying patterns, and assessing financial health using advanced ADF workflows. By solving these problems, you'll gain experience in designing end-to-end data solutions, working with diverse datasets, and generating actionable insights.