Mule Programming Style Guide: Project Structure

Project structure is important! In this post I review how I layout my Mule microservices projects, as well as other types of projects I've build with Mule.

Mule Programming Style Guide: Project Structure

Introduction

This is Part 3 of my N-Part series documenting my Mule programming style. If you haven't checked out Part 1, A Simple Main Flow, and Part 2, DataWeave Code check them out! This post will cover how I structure my Mule projects. This posts assumes Mule 4.

A well-organized project can help reap the same benefits of well-organized code. Just like organizing your code well can help future readers of your code more quickly identify what your code is supposed to do, a well-organized project can help future readers of your code easily find what they're looking for. This becomes more and more true as the project grows in size. Luckily, we don't have to worry about this issue too much, because when we're developing with Mule, we're typically creating microservices or small integrations. That said, it's still something we should concern ourselves with, because we can get a lot of benefit for not a lot of effort.

Attributes of the project itself will typically determine the best way to organize your code, and I find that Mule is no different. I find that a Mule project falls into either one of two camps: microservice, or not. This is obviously an over-simplification, so you may want to develop your own rules to handle other cases that apply to your organization. The structure of your project won't only be determined by what kind of project it is, but other factors as well. You may decide to organize your projects a certain way because you know parts of them will be temporary, or may be broken out into another application later. You may have a client that decided to purchase Mule, but has decided it's not in their financial interest at the time-being to implement microservices. You should account for these scenarios as they appear, but the information in this post should give you a good base to start off from.

There are so many ways you can skin this cat, and frankly, it's not that important that you follow the guidelines below. However, it is incredibly important that your organization agrees on a common set of rules to structure similar projects and that these rules are enforced through code analysis tools or code reviews / pull requests. This will greatly decrease the amount of time it takes a developer to become familiar with a new project. You should absolutely create a template project and put it up in Exchange to make sticking to these rules as easy as possible.

Organizing Microservice Projects

The src/main/app Directory

Starting with the src/main/app directory, I typically have 3 files:

  1. config.xml or global.xml (don't matter which, but pick one and stick with it)
  2. api.xml
  3. process.xml

config.xml contains all of my global configuration elements. So all of my properties placeholders, DB configs, HTTP listener configs, global functions, etc., all go in this file. If I'm using a global exception strategy for my project, it goes here as well. If there is a situation where flows are shared among multiple files, I typically place them here as well. The only thing I'm really flexible on here is the HTTP listener generated by Anypoint Studio when you create a new project with API Kit. It can make sense to leave the HTTP listener in the api.xml file because it's not going to be used by any other component in the application. Again, it doesn't really matter what you decide to do, just be consistent.

api.xml contains all the information generated by Anypoint Studio from your RAML API definition. Since this file is susceptible to be modify by Anypoint Studio, you should minimize the amount of custom code you place here. The only thing this file should contain is the auto-generated code, and any flow-refs to your process.xml flows. If you choose to modify the generated exception handler, you should move that into its own file as well. This makes it really easy to re-generate this code if your RAML changes. You don't have to worry about moving around a bunch of code, you just add your flow-refs and you're ready to go.

process.xml contains all of my application logic. All the flow-refs from api.xml point here. Sometimes, this file can get a little unruly. If I'm stacking up a lot of flows in here, and am getting the feeling it would be best to break things up, I rename this flow to process-<functionality>.xml and add another one with the same schema. This gives a kind of psuedo-namespacing that makes it clear that both files contain application logic, though what they cover is different.

Occasionally, I'll get a requirement for something that doesn't fit into the above layout. In that case, I almost always create a new file. For example: the client wants to put the cron triggers in the application (Is that the best place for them? Not likely. Is the client insistent that they go there anyway? Of course they are). So I add a triggers.xml file, and put my pollers and their respective flow-refs in there. If they decide they want to pull the triggers into another application, I just delete the file.

I'll add other cases as they come up, but triggers.xml is the most common, in my experience.

The src/main/resources Directory

The main things I keep in my src/main/resources directory are:

  1. The log4j2.xml config
  2. Any environment-specific properties files
  3. Any API-related documents (RAML, example files, etc.)
  4. Any reuseable scripts or DataWeave modules.
  5. Pretty much anything else that isn't Java code and doesn't make sense in src/main/app

I keep all my log4j2.xml and environment-specific properties files in this directory, and that doesn't change regardless of what kind of project I'm working with, so I won't bring it up again in this post.

Any API-related documents go under src/main/resources/api (this applies to Mule 4 projects only. Mule 3 automatically puts API-related files in src/main/api, so I stay with that convention). At the root of this directory, I have the RAML API definition, which I typically name api.raml (some people prefer something much more specific, again, just be consistent with what you chose to do across your organization). Any sub-directories are reserved for information as it relates to the API. Things like RAML DataTypes, examples, etc., will all go in their own directories. This makes it easy to zip the entire API definition and send it off, if necessary, without needing to move things around.

Any reuseable scripts or DataWeave modules go under src/main/resources/scripts. If I'm using a module, I'd put it under scripts/modules, otherwise, I'd leave it under scripts unless there's a compelling reason to organize further.

Organizating Other Types of Projects

Depending on what your project is supposed to accomplish, this can get rather tricky, but there are a few rules I stick to.

The src/main/app Directory

Starting with the src/main/app directory, I typically have one of two kinds files:

  1. config.xml
  2. *.xml

As specified above, all of my configurations go in my config.xml. Please read above if you haven't already and are looking for more details.

*.xml, helpful, right? I put this here because what you name the rest of your Mule configuration files is going to be highly dependent on what kind of application you're developing. As a general rule of thumb, files that cover similar functionality should be named similarly, using the psudo-namespacing that I covered earlier, and different functionality all together should be expressed with a separate file.

As an example, I recently worked on a project where the application would pull from the queue, route the message to series of mappings based the message content, and then push that file to another queue. In this case, I had my config.xml file, like I always do. But I also had a router.xml file, which contained the code to read the message from the queue, and route it to a particular mapping. I started out with only two mappings I could possibly route to, so I put them in the same file called mapping.xml. As the number of mappings grew over time, I decided to break them out and name them for the systems they were being mapped to like mapping-billing.xml and mapping-invoices.xml. This makes it easier for future developers who are say, looking for a bug in the billing mapping, to find what file the relevant code is in. Any components that were shared between the two mappings were placed in config.xml.

Additional Considerations

The project layout that you chose now will greatly influence how future developers modify your project. It might seem like overkill to create a new xml file with a single flow, but if that same file has a descriptive name, and can cue another developer that they should put a certain type of code in that file, your project is going to maintain its structure better over time, instead of accumulating spaghetti code in a few files. Think about the triggers example from earlier. If a developer is tasked with creating a new cron trigger, and they see a file called triggers.xml with a single flow containing a poller and flow-ref, it's pretty likely they will put their code there. However, if that file didn't exist, I'd bet money it ends up in process.xml.

Conclusion

Your project structure, just like your code, can go a long way in helping other developers quickly make necessary modifications to your code. When dealing with a microservice project, you can typically use the 3-file layout of config, process, and api xml files. When you stray from this approach, it's important to be more selective in how you name your files and how to separate your code. Give your files descriptive names, while giving files with similar functionalities similar names. Stay tuned next time where I analyze the often-abused choice component.