ForEach Activity

The ForEach activity in Azure Data Factory is a powerful tool that enables you to perform iterative operations, such as processing multiple files or iterating through a list of items.

The Lookup activity is often used in combination with the ForEach activity to retrieve a list of items, such as file names or table names, for iteration. By passing the Lookup output to the ForEach activity, you can dynamically process each item in the collection, enabling scalable and flexible workflows in Azure Data Factory.

How to Use ForEach Activity

1. Start by using a Lookup activity to retrieve data;
2. Drag the ForEach Activity from the Activities pane onto the pipeline canvas;
3. In the Settings tab of the ForEach Activity, set the Items field to the output of the Lookup activity, such as: @activity('LookupActivityName').output.value;
4. Inside the ForEach Activity, add a Copy Data Activity or any other activity to process each item in the list dynamically;
5. In the Source tab of the Copy Data Activity, configure the input dataset for the data you want to process and use dynamic content to reference the current iteration. In our case we use the following query: SELECT * FROM CovidData WHERE "HHSRegion" = '@{item().HHSRegion}';
6. In the Sink tab of the Copy Data Activity, configure the destination table name dynamically using the expression: TableName_@{item().ColumnName};
7. Validate and test the pipeline to ensure that data is correctly processed for each item in the dataset.

@item() expression

In Azure Data Factory (ADF), the @item() expression is used inside the ForEach activity to reference the current item in the loop as it iterates through the collection of items. The @item() expression allows you to dynamically access data or values from the current iteration and use them in subsequent activities within the ForEach loop.

Its attributes refer to the properties of the item being iterated over. For example, if the loop is iterating through a list of objects, @item() can access specific attributes like @item().Region or @item().FileName, allowing dynamic referencing of data for each iteration.