Mastering DataWeave 2.0: How to Flatten Deeply Nested JSON Arrays

We have all been there. You trigger an API call and receive a JSON payload that resembles a set of Russian Nesting Dolls—arrays inside arrays inside arrays. In the world of MuleSoft integration, handling this specific type of "Inception-style" hierarchy can be a major headache if you do not have the right tools in your belt.

Today, we are going to explore how to use DataWeave 2.0 (dwl) to elegantly solve this problem. We will take a messy, arbitrarily deep structure and "flatten" it into a clean, simple list using a recursive function.

The Problem: "Inception" Style Data

Imagine you have a JSON structure where your actual data is hidden inside layers of a children array. The tricky part? The nesting depth isn't fixed. It could be two levels deep, or it could be twenty. Your goal is to extract specific data points, regardless of how deep they are buried.

Here is the source data we need to tackle. Notice how the names "Sandy," "Bob," and "Frank" are buried at completely different levels?

Input JSON

{
  "children": [
    {
      "children": [
        {
          "children": [
            {
              "children": [
                {
                  "children": [
                    {
                      "children": [
                        { "field": 1, "name": "Sandy" },
                        { "field": 2, "name": "Alex" }
                      ]
                    },
                    { "field": 3, "name": "Bob" }
                  ]
                },
                { "field": 4, "name": "Charlie" }
              ]
            },
            { "field": 5, "name": "David" }
          ]
        },
        { "field": 6, "name": "Eve" }
      ]
    },
    { "field": 7, "name": "Frank" }
  ]
}

The Goal: We want to ignore the hierarchy completely. We just want to grab every object that contains a field and a name, returning them in a single, flat list.

The Solution: A Recursive DataWeave Function

To solve this, we do not need complex for-loops or messy indexing. We need recursion. In programming terms, a recursive function is simply a function that calls itself until it finishes a specific task.

The script below defines a function called flattenChildren. It automatically detects if it needs to dig deeper or if it has found the treasure.

DataWeave Script

%dw 2.0
output application/json
fun flattenChildren(data) =
  if (data is Array)
    data flatMap (item) -> flattenChildren(item)
  else if (data.children?)
    flattenChildren(data.children)
  else if (data.field?)
    [data]
  else
    []
---
flattenChildren(payload.children)

How It Works: The Logic Breakdown

This script works like a search party sweeping through a building, room by room. Here is what is happening under the hood:

• The Engine (flattenChildren): This function accepts your data object and makes a decision based on the "shape" of the data it receives.
• Handling Arrays (is Array): If the function finds an array, it knows there are multiple items to check. It uses flatMap to iterate through every item.
  Why flatMap? Crucially, flatMap takes the results and "flattens" them into one single array. If we used standard map, we would end up with arrays inside arrays, which is exactly what we are trying to avoid.
• Diving Deeper (data.children?): If the function finds a property called "children," it knows it hasn't hit the bottom yet. It calls itself (recursion) to look inside that child object.
• Found It! (data.field?): This is our "Eureka" moment. If the function finds an object with a field property, it knows this is the data we want. It wraps that object in brackets [data] to make it an array and returns it.
• The Base Case (else []): If the data is not an array, has no children, and has no field, the function returns an empty array []. This effectively discards irrelevant data so your final output stays clean.

Why This Approach Wins

By using this recursive pattern, you gain three major advantages:

1. Conciseness: We achieved complex logic in just a few lines of code.
2. Flexibility: Because it is recursive, it does not matter if the nesting is 2 levels deep or 20 levels deep—the code works exactly the same without modification.
3. Readability: DataWeave is designed to be functional and readable, making it easier for other developers to maintain your integration later.

By combining recursion with flatMap, you turn a complex hierarchy into a manageable list, ready for the next step in your integration process.