How to Identify Duplicate Values in an Array with Mule 4 DataWeave 2

Imagine you are organizing a chaotic bag of numbered lottery balls. Some numbers appear only once (the lucky unique ones), while others pop up multiple times. To make sense of this data, you need to sort them into two distinct buckets: one for the "One-Hit Wonders" (unique numbers) and one for the "Repeats" (duplicates).

In the world of MuleSoft integration, developers often face this exact scenario with raw data arrays coming from APIs or databases. Today, we are going to look at a Mule 4 DataWeave 2 script that automatically sorts your data, counts the occurrences, and organizes everything into a clean, readable JSON format.

The Goal

We want to take a simple list of numbers, analyze how often each number appears, and separate them into two groups:

• Duplicates: Numbers that appear more than once.
• Non-Duplicates: Numbers that appear exactly once.

---

Phase 1: The Setup

First, we need to establish the environment. We tell the DataWeave engine what version we are using and how we want the output formatted. We use %dw 2.0 to define the version and set the output to application/json so it is easy for humans and downstream systems to read.

We also import the core Arrays module. While we might not strictly need it for basic grouping, it's best practice to have standard tools available when working with lists.

%dw 2.0
output application/json
import * from dw::core::Arrays

---

Phase 2: The Input Data

Here is our "bag of balls." It is a standard variable containing an array of integers. Notice that numbers like 1 and 5 appear multiple times, while numbers like 3 and 4 are unique.

var inputData = [
 3, 1, 5, 4, 2, 1, 5, 6, 7, 1
]

---

Phase 3: The Logic Explained

This is where the magic happens. We aren't just counting; we are restructuring the data in three specific logic steps:

1. Group and Count: We use groupBy ($) to cluster identical numbers together. Then, we use pluck to transform those clusters into an object that lists the specific value and the count (size) of that group.
2. Filter by Frequency: We check the size of the group using sizeOf($).
3. Categorize: Finally, we group the results again based on whether the count is greater than 1. We use mapObject to rename these boolean groups into friendly labels: "duplicate" or "non-duplicate".

Here is the logic block isolated:

inputData groupBy ($) pluck(
 if(sizeOf($)>1){
 "value" : $$,
 "count" : sizeOf($)
    }
 else{
 "value" : $$,
 "count" : sizeOf($)
    }
) groupBy ( $.count > 1 ) mapObject(
 if($$ ~= true)(
 "duplicate" : $
    )
 else(
 "non-duplicate" : $
    )
)

---

Phase 4: The Complete Script

When you put it all together, you get a robust script that takes raw, messy data and transforms it into structured intelligence.

Copy and paste the code below into your MuleSoft Transform Message component:

%dw 2.0
output application/json
import * from dw::core::Arrays
var inputData = [
 3, 1, 5, 4, 2, 1, 5, 6, 7, 1
]
---
inputData groupBy ($) pluck(
 if(sizeOf($)>1){
 "value" : $$,
 "count" : sizeOf($)
    }
 else{
 "value" : $$,
 "count" : sizeOf($)
    }
) groupBy ( $.count > 1 ) mapObject(
 if($$ ~= true)(
 "duplicate" : $
    )
 else(
 "non-duplicate" : $
    )
)

---

Conclusion

By chaining groupBy, pluck, and mapObject, we successfully transformed a flat list of numbers into a categorized report. This pattern is essential for data validation in integration flows, allowing you to quickly isolate duplicated records before processing them further.