DataWeave - Transforming an Array to an Object

How to transform Arrays into Objects in DataWeave

DataWeave - Transforming an Array to an Object

When it comes to transforming between arrays and objects we have four different permutations:

Array  -> Array
Object -> Object
Object -> Array
Array  -> Object

Depending on your experience, you may have picked up that there are a handful of functions that you will typically reach for in these situations. They are:

[map/filter] Array  -> Array
[mapObject]  Object -> Object
[pluck]      Object -> Array
[groupBy?]   Array  -> Object

What I mean is this: if you know a transformation is going to modify values in an array and return an array you'd probably reach for map first, if you know a transformation is going to modify values of an object, and return an object, you'd reach for mapObject first, etc.

But things tend to get blurry when you need to transform an array into an object. What function do you reach for? It might strike you to scroll through the documentation here and see if you can find a function with a signature like (':array', ':function') => ':object' (meaning the function has one input of type :array, another input of type :function, and returns one value of type :object). groupBy is a viable candidate based on its signature but it's not general enough to do what we want. What we can use instead is a function that does something more general, the reduce function. You'll notice that the reduce function has the signature (':array', ':function') => ':any'. :object is a subtype of :any so this is a viable candidate for our use case.

A Simple Example

If you recall from my previous blog post on reduce, you can use reduce to transform an array into whatever you want, a single value, a collection of values, a collection of collections, etc. Therefore, you can use reduce to transform an array into an object. Let's check it out.

%dw 1.0
%output application/json

%var arr = [1, 2, 3, 4, 5]
---
arr reduce ((v, obj={})
  obj ++ {(ordinalize v): v})

And this will output:

{
  "1st": 1,
  "2nd": 2,
  "3rd": 3,
  "4th": 4,
  "5th": 5
}

This is iterating over each value of the array, and accumulating the result of {(ordinalize v): v} into an object. After first pass the object, obj, is

{
  "1st": 1
}

after the second pass obj is

{
  "1st": 1,
  "2nd": 2
}

and so on, until the array has no more values to iterate over.

I've found this pattern is useful for when I have an array of objects that I'd rather access as a key:value pair. For example:

%dw 1.0
%output application/json

%var arr = [
  {dev:  "c6824476-b7e2-4e8a-9281-bdc40663bb93"},
  {test: "b1749c38-1e25-42f7-b928-43041c80b496"},
  {qa:   "eda249c3-d85a-4031-bbda-88efd30bba8c"},
  {prod: "7f44b450-18ff-45fa-85f3-8ef5c82b4989"}
]
---
arr reduce ((env, obj={}) -> obj ++ env)

Which will output:

{
  "dev":  "c6824476-b7e2-4e8a-9281-bdc40663bb93",
  "test": "b1749c38-1e25-42f7-b928-43041c80b496",
  "qa":   "eda249c3-d85a-4031-bbda-88efd30bba8c",
  "prod": "7f44b450-18ff-45fa-85f3-8ef5c82b4989"
}

If you prefer the shorthand, arr reduce ($$ ++ $) will work as well.

A More Complex Example

Below is a more complex example of the above pattern. We have a list of values that we'd prefer to access as a single entity. Here's our input:

[
  {
    "name": "size"
    "options: [
      {
        "value": 12345,
        "label": "small"
      },
      {
        "value": 23456,
        "label": "medium"
      },
      {
        "value": 34567,
        "label": "large"
      }
    ]
  },
  {
    "name": "fabric"
    "options: [
      {
        "value": 45678,
        "label": "cotton"
      },
      {
        "value": 56789,
        "label": "silk"
      },
      {
        "value": 67980,
        "label": "polyester"
      }
    ]
  }
]

And we need this as the output:

{
  "size": {
    "small":  12345,
    "medium": 23456,
    "large":  34567
  },
  "fabric": {
    "cotton":    45678,
    "silk":      56789,
    "polyester": 67890
  }
}

Let's check out how we might solve this:

%dw 1.0
%output application/java

%var input = <see_above>
---
input reduce ((value, parent={}) -> 
  parent ++ {
    (value.name as :string): value.options reduce ((option, child={}) ->
      child ++ { (option.label as :string): option.value 
    })
  }
)

In this solution, we end up nesting a reduce inside of another reduce. If we look back at our original data, this makes sense. We have the outermost array that we want to compress into an object, and within each object in the outermost array, there is another array we need to compress into an object.

Wrapping it up

Let's take a step back and look at all of our examples. There's a pattern, here: we're starting with an empty object, and populating that object with key:value pairs using each value from the input array:

arr reduce ((v, obj={}) ->
  obj ++ {<something_to_get_a_key>: <something_to_get_a_value>})

Where <something_to_get_a_key> and <something_to_get_a_value> will likely be replaced with some kind of manipulation of v, but not always (e.g. flattening an array of objects into a single object). That's really all there is to it. As always, feel free to send me a message on LinkedIn if you have any questions.