Mastering DataWeave: 10 Real-World Transformation Scenarios (Part III)

by MuleTrains •
0

Welcome back to the third installment of our DataWeave deep dive! If you’ve been following along, you know that DataWeave is the absolute engine room of MuleSoft’s data transformation capabilities. Today, we aren't just doing simple field mappings; we are tackling the edge cases, the dynamic logic, and the "how do I even do that?" moments.


Whether you are prepping for a technical interview or you are currently stuck on a nasty integration problem, these 10 scenarios cover the tricky stuff—like parsing unstructured text, generating dynamic XML namespaces, and implementing custom recursion.

Let's dive into the code.

Scenario 1: Filtering Specific IDs from Text Data

The Challenge: You have an input array that looks like JSON, but be careful—sometimes the input type is treated as text. Your goal is to extract only the objects where the sid equals 123.

Input:

[
  { "sid": 123, "sfname": "sai" },
  { "sid": 123, "slname": "kumar", "snum": 9087654132 },
  { "sid": 124, "sfname": "ravi", "slname": "kanth" },
  { "sid": 125, "sfname": "hari" }
]

Target Output:

{
  "sid": 123,
  "sfname": "sai",
  "sid": 123,
  "slname": "kumar",
  "snum": 9087654132
}

Solutions: Here are three ways to skin this cat. You can use standard filtering, takeWhile, or partition.

Script 1 (The Filter Approach)

%dw 2.0
output application/json
---
{(payload filter($.sid ~= 123))}

Script 2 (Using Arrays Library)

%dw 2.0
output application/json
import * from dw::core::Arrays
---
{(payload takeWhile ((item) -> item.sid ~= "123"))}

Script 3 (Partitioning)

%dw 2.0
output application/json
import * from dw::core::Arrays
---
{((payload partition  ((item) -> (item.sid ~= "123"))).success)}

Scenario 2: Parsing Unstructured Q&A Text

The Challenge: You are given a raw string containing questions and answers mashed together. Hint: Look for the punctuation. We split by statements (.) and questions (?).

Input:
Did not find your language?Propose yourself to translate our contents by sending an email with your references and your suggestion.What are the origins of Lorem Ipsum?Contrary to what one might think, the Lorem ipsum text, despite being meaningless, has noble origins.

Target Output: A structured JSON array of Question/Answer objects.

Script 1 (String Splitting)

%dw 2.0
output application/json
import * from dw::core::Strings
---
payload splitBy  "." map{
        question : ($ splitBy  "?")[0],
        answer : ($ splitBy  "?")[1]
    }

Script 2 (Regex Splitting)

%dw 2.0
output application/json
import * from dw::core::Arrays
---
payload splitBy /[\.\?]/ divideBy  2 map do{
 var eachDetails = $
 ---
    {
 "question" : eachDetails[0],
 "answer" : eachDetails[1]
    }
}

Scenario 3: Dynamic XML Tags with Namespaces

The Challenge: This requires a dynamic approach. You need to generate an XML structure where the Root tag and parent tag names are dynamic (variables), and you must use a specific namespace.

Input:

{
  "intA": "40",
  "intB": "50",
  "intC": "12"
}

Solutions:

Script 1 (Namespace as Object)

%dw 2.0
output application/xml
var operation="Divide"
var namespace = {uri: "http://tempuri.org/", prefix: "ns0"} as Namespace
---
namespace#"$(operation)" : payload mapObject{
    namespace#"$($$)" : $
}

Script 2 (Explicit Namespace Declaration)

%dw 2.0
output application/xml
var operation="Divide"
ns ns0 http://tempuri.org/
---
ns0#"$(operation)" : payload mapObject{
    ns0#"$($$)" : $
}

Scenario 4: The "Hard" One - Numbers to Words

The Challenge: This is a logic-heavy problem often used to test deep coding skills. You need to convert a numeric string into its English word representation using the Indian numbering system (Crores/Lakhs).

Input: 92345324
Target Output: "nine crore twenty three lakh forty five thousand three hundred twenty four"

Solution:

%dw 2.0
output application/json
import * from dw::core::Strings

var oneToNine = {
"1": "one",
"2": "two",
"3": "three",
"4": "four",
"5": "five",
"6": "six",
"7": "seven",
"8": "eight",
"9": "nine"
}

var tenTotwenty = {
"1": "eleven",
"2": "twelve",
"3": "thirteen",
"4": "fourteen",
"5": "fifteen",
"6": "sixteen",
"7": "seventeen",
"8": "eighteen",
"9": "nineteen"
}

var secondposetion = {
"2": "twenty",
"3": "thirty",
"4": "forty",
"5": "fifty",
"6": "sixty",
"7": "seventy",
"8": "eighty",
"9": "ninty"
}

var remaining = {
"3":"hundred",
"4":"thousand",
"5":"lakh",
"8":"crore"
}

fun convertToSChar(num :Number) = (

do{

var tmpData = []

var size = sizeOf(num)

var single = if(size == 1) tmpData << oneToNine[num[-1]] else []

var ten = flatten(tmpData << (if(num[-2] ~= 1) tenTotwenty[num[-1]] else (tmpData << secondposetion["$(num[-2])"] << oneToNine[num[-1]]) )) default []

var hundred = (tmpData << oneToNine[num[-3]] << remaining["$(sizeOf(num[-3 to-1]))"]) default []

var thousand = (tmpData << oneToNine[num[-4]] << remaining["$(sizeOf(num[-4 to-1]))"]) default []

var thousands = flatten(tmpData << (if(num[-5] ~= 1) tenTotwenty[num[-4]] else ( secondposetion["$(num[-5])"]) )) default []

var lack = (tmpData << oneToNine[num[-6]] << remaining["$(sizeOf(num[-5 to -1]))"]) default []

var lacks = flatten(tmpData << (if(num[-7] ~= 1) tenTotwenty[num[-6]] else ( secondposetion["$(num[-7])"]) )) default []

var crore = flatten(tmpData << tmpData << oneToNine[num[-8]] << remaining["$(sizeOf(num[-8 to -1]))"] ) default []

---
flatten([crore,lacks,lack,thousands,thousand,hundred,ten,single])
}
)
---
if(payload == 0) "zero"
else if(isEmpty(payload)) "please pass the number"
else convertToSChar(payload) joinBy " "

Scenario 5: Recursive Summation

The Challenge: Sum all numbers in an array. Constraint: Do not use the built-in sum() function. You must implement a recursion or reduction approach.

Input: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Target Output: 55

Script 1 (Cheating? No, just standard sum - for reference)

%dw 2.0
output application/json
---
sum(payload)

Script 2 (The Reduction Method)

%dw 2.0
output application/json
---
payload reduce($$ + $)

Script 3 (SumBy Approach)

%dw 2.0
output application/json
import * from dw::core::Arrays
---
payload sumBy ((param_1) -> param_1 )

Scenario 6 & 7: Dynamic Start and End Calculations

The Challenge: Here we have two variations. You need to calculate "Start" and "End" fields based on the input values, their position in the array (index), and mathematical operations like Modulo.

Input:

[
  { "ID": 1234, "Name": "salary Account" },
  { "Name": "personal", "ID": 987 },
  { "Name": "Loan Account", "ID": 654 }
]

Logic Set A (Simple Size Calculation)

Script 1:

%dw 2.0
output application/json
---
payload map{
    ID : $.ID,
    Name : $.Name,
    Start : sizeOf($.ID),
    End : sizeOf($.Name) - sizeOf($.ID) + $$
}

Script 2 (Using Concatenation):

%dw 2.0
output application/json
---
payload map (
    $ ++ {
    Start : sizeOf($.ID),
    End : sizeOf($.Name) - sizeOf($.ID) + $$
})

Logic Set B (Complex Modulo Arithmetic)

Script 1:

%dw 2.0
output application/json
---
payload map(
 if(($$ mod 2) == 0){
        ID : $.ID,
        Name : $.Name,
        Start : sizeOf($.ID),
        End : $.ID/2 - (sizeOf($.Name) + sizeOf($.ID) + $$ )
    }
 else  {
        Name: $.Name,
        ID: $.ID,
        start: sizeOf($.ID),
        end: $.ID/2 + (sizeOf($.Name) + sizeOf($.ID)) + $$
  }
)

Script 2:

%dw 2.0
output application/json
---
payload map(
    $ ++ (if(($$ mod 2) == 0){
        Start : sizeOf($.ID),
        End : $.ID/2 - (sizeOf($.Name) + sizeOf($.ID) + $$ )
    }
 else  {
        start: sizeOf($.ID),
        end: $.ID/2 + (sizeOf($.Name) + sizeOf($.ID)) + $$
  })
)

Scenario 8: XML to JSON (Preserving Attributes)

The Challenge: Convert an XML catalog of books into JSON. Constraint: You must retain the attributes (like @id) in the final JSON structure, which typically gets lost in standard conversions.

Input:

<catalog>
 <book id="bk101">...</book>
 <book id="bk112">...</book></catalog>

Solutions:

Script 1 (The "WriteAttributes" Flag)

%dw 2.0
output application/json writeAttributes=true
---
payload

Script 2 (Manual Attribute Mapping)

%dw 2.0
output application/json
---
payload mapObject ((value, key, index) -> 
(key) : value mapObject ((bookVal, bookKey, bookIdx) -> 
(bookKey) : (if(bookKey.@?)(
    bookKey.@ mapObject(
        ("@" ++ $$) : $
    )
) else {} ) ++ bookVal
)
)

Scenario 9: Creating Keys from Value Characters

The Challenge: Transform the input so the JSON Key is dynamically derived from the Value. Rule: The new Key should be the First Character + the Last Character of the Name field.

Input:

{
 "Tablelist": [
    { "Name": "nemalidinne_satish_reddy" },
    { "Name": "nemalidinne_ANIL_reddk" },
    ...
  ]
}

Solutions:

Script 1 (Map and Concat)

%dw 2.0
output application/json 
---
{(payload.Tablelist.Name map(
    ($[0] ++ $[-1]) : $
))}

Script 2 (MapObject)

%dw 2.0
output application/json 
---
{(payload.Tablelist)} mapObject(
    ($[0] ++ $[-1]) : $
)

Script 3 (Pluck)

%dw 2.0
output application/json 
---
{({(payload.Tablelist)} pluck(
    ($[0] ++ $[-1]) : $
))}

Scenario 10: Sorting, Grouping, and Indexing

The Challenge: You have a list of products with expiration dates and prices. Hint: You need to sort the data, group it by product name, and then add a specific item index counter for each group.

Input:

[
  { "product name": "bread", "ExpairyDate": "11/01/2011", "Price": 40 },
  { "product name": "Apple", "ExpairyDate": "22/12/2012", "Price": 60 },
  { "product name": "milk", "ExpairyDate": "12/11/2023", "Price": 60 },
  ...
]

Solutions:

Script 1 (GroupBy, Pluck, OrderBy)

%dw 2.0
output application/json 
---
flatten(payload groupBy $."product name" pluck $  orderBy(-sizeOf($)) map(
    $ map ((item, index) -> {
 "product name": item."product name",
 "item": index + 1,
 "ExpairyDate": item.ExpairyDate,
 "Price": item.Price
  }
    )
))

Script 2 (Concise Mapping)

%dw 2.0
output application/json 
---
flatten(payload groupBy ($."product name") orderBy(-sizeOf($)) pluck(
    $ map(
        $ ++ {
            item : $$ + 1
        }
    )
))

Script 3 (Using ValuesOf and FlatMap)

%dw 2.0
output application/json 
---
valuesOf(payload groupBy $."product name") orderBy(- sizeOf($)) flatMap(
    $ map(
        $ ++ {
 "item" : $$ + 1
        }
    )
)

Conclusion

These scenarios cover a wide range of DataWeave functionality, from basic filtering to advanced recursion and complex object mapping. Mastering these patterns is essential for any MuleSoft developer handling real-world data integration.

Tags:
Post a Comment (0)
Previous Post Next Post