Marcin bloguje

Complex flows with Apache Camel

At work, we're mainly integrating services and systems, and since we're on a constant lookout for new, better technologies, ways to do things easier, make them more sustainable, we're trying to

Usually we use Apache Camel for this task, which is a Swiss-knife for integration engineer. What's more, this tools corresponds well with our approach to integration solutions:

• try to operate on XML messages, so you get the advantage of XPaths, XSL and other benefits,
• don't convert XML into Java classes back and forth and be worried with problems like XML conversion,
• try to get a simple flow of the process.

However, at first sight Apache Camel seems to have some drawbacks mainly in the area of practical solutions ;-). It's very handy tool if you need to use it as a pipeline with some marginal processing of the data that passes through it. It gets a lot harder to wrap your head around if you consider some branching and intermediate calls to external services. This may be tricky to write properly in Camel's DSL.

Here is a simple pipeline example:

And here the exact scenario we're discussing:

What I'd like to show is the solution to this problem. Well, if you're using a recent version of Camel this may be easier, a little different, but should still more-or-less work this way. This code is written for Apache Camel 1.4 - a rather antic version, but that's what we're forced to use. Oh, well.

Ok, enough whining!

So, I create a test class to illustrate the case. The route defined in TestRouter class is responsible for:

1. receiving input
2. setting exchange property to a given xpath, which effectively is the name of the first XML element in the input stream
3. than, the input data is sent to three different external services, each of them replies with some fictional data - notice routes a, b and c. The SimpleContentSetter processor is just for responding with a given text.
4. the response from all three services is somehow processed by RequestEnricher bean, which is described below
5. eventually the exchange is logged in specified category

Here is some code for this:

public class SimpleTest {
    public void setUp() throws Exception {
        TestRouter tr = new TestRouter();
        ctx.addRoutes(tr);
    }

    @Test
    public void shouldCheck() throws Exception {
        ctx.createProducerTemplate().send("direct:in", getInOut("<a/>"));
    }


    class TestRouter extends RouteBuilder {

        public void configure() throws Exception {

            ((ProcessorType<ProcessorType>)from("direct:in")
            .setProperty("operation").xpath("local-name(/*)", String.class)
            .multicast(new MergeAggregationStrategy())
                .to("direct:a", "direct:b", "direct:c")
            .end()
            .setBody().simple("${in.body}"))
            .bean(RequestEnricher.class, "enrich")
            .to("log:pl.touk.debug");
            
            from("direct:a").process(new SimpleContentSetter("<aaaa/>"));
            from("direct:b").process(new SimpleContentSetter("<bbbb param1=\"1\" param2=\"2\" param3=\"3\"/>"));
            from("direct:c").process(new SimpleContentSetter("<cccc/>"));
        }
    }
}

What's unusual in this code is the fact, that what normally Camel does when you write a piece of DSL like:

	.to("direct:a", "direct:b", "direct:c")

is pass input to service a, than a's output gets passed to b, becomes it's input, than b's output becomes c's input. The problem being, you loose the output from a and b, not mentioning that you might want to send the same input to all three services.

That's where a little tool called multicast() comes in handy. It offers you the ability to aggregate the outputs of those services. You may even create an AggregationStrategy that will do it the way you like. Below class, MergeAggregationStrategy does exactly that kind of work - it joins outputs from all three services. A lot of info about proper use of AggregationStrategy-ies can be found in this post by Torsten Mielke.

public class MergeAggregationStrategy implements AggregationStrategy {

	public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
		if (oldExchange.isFailed()) {
			return oldExchange;
		}
		transformMessage(oldExchange.getIn(), newExchange.getIn());
		transformMessage(oldExchange.getOut(), newExchange.getOut());
		return newExchange;
	}
	
	private void transformMessage(Message oldM, Message newM) {
		String oldBody = oldM.getBody(String.class);
		String newBody = newM.getBody(String.class);
		newM.setBody(oldBody + newBody);
	}
	
}

However nice this may look (or not), what you're left with is a mix of multiple XMLs. Normally this won't do you much good. Better thing to do is to parse this output in some way. What we're using for this is a Groovy :). Which is great for the task of parsing XML. A lot less verbose than ordinary Java.

Let's assume a scenario, that the aggregated output, currently looking like this:

is to be processed with the following steps in mind:

• use <aaaa/> as the result element
• use attributes param1, param2, param3 from element <bbbb/> and add it to result element <aaaa/>

public class RequestEnricher {
	
	public String enrich(@Property(name = "operation") String operation, Exchange ex) {
		
		use(DOMCategory) {
			def dhl = new groovy.xml.Namespace("http://example.com/common/dhl/schema", 'dhl')
			def pc = new groovy.xml.Namespace("http://example.com/pc/types", 'pc')
			def doc = new XmlParser().parseText(ex.in.body)
			
			def pcRequest   = doc."aaaa"[0]
			
			["param1", "param2", "param3"].each() {
				def node = doc.'**'[("" + it)][0]
				if (node)
					pcRequest['@' + it] = node.text()
			}
			
			gNodeListToString([pcRequest])
		}
		
	}
	
	String gNodeListToString(list) {
		StringBuilder sb = new StringBuilder();
		list.each { listItem ->
			StringWriter sw = new StringWriter();
			new XmlNodePrinter(new PrintWriter(sw)).print(listItem)
			sb.append(sw.toString());
		}
		return sb.toString();
	}
	
}

What we're doing here, especially the last line of enrich method is the conversion to String. There are some problems for Camel if we spit out Groovy objects. The rest is just some Groovy specific ways of manipulating XML. But looking into enrich method's parameters, there is @Property annotation used, which binds the property assigned earlier in a router code to one of the arguments. That is really cool feature and there are more such annotations:

• @XPath
• @Header
• @Headers and @Properties - gives whole maps of properties or headers

This pretty much concludes the subject :) Have fun, and if in doubt, leave a comment with your question!

Meetbsd 2010

Some time ago, I've attended MeetBSD conference in Kraków. This BSD event is held yearly in either Warsaw, or Kraków. Due to relatively small group of people that registered there was only one track, which had both good and bad sides - you didn't have to choose from myriads of lectures, but there was no way to skip boring ones either. Well, I guess this kind of niche conference - about operating system :) - will not attract bigger attention.

6+ hour layover in Stockholm

Last Monday, while coming home from Sweden, I had ~7h layover between my flights. Since Arlanda Airport does not offer a lot to do, at least not for me and not for 7h, I had decided to go to Stockholm. This post sums up a few nice places, that I'd like to visit on my next trip there.

I took Arlanda Express, fast train that takes you directly from the airport to Stockholm. On board, you travel with 200km/h in good enough conditions. The trains takes you near the Central station in 20 minutes, which is great! The main venue I planned to see was Gamla Stan. All the attractions were also there, on the island.

The first noteworthy place on the road was Christina's costume salon. Unfortunately closed on the day of my visit, however since it was recommended to me, and the place look interesting I plan on paying it another visit. This shop offers a lot of costumes, different sizes, themes. Website is: Shop's page

Seems like just a usual gift shop, but has this unique feeling, this Norsk spirit. Shop offers many fantastic gifts and souvenirs, but unfortunately they don't come cheap. Nevertheless, worth visiting and perhaps buying. Especially nice were little metal statues of Viking gods. Handfaste - The Viking Shop

Neat little workshop, that does metal signs on request, from your design. Very good quality. Actually there are quite a few of such workshops nearby, but this may serve as a reminder. Det "lille" Skyltmakerie

Awesome sci-fi, fantasy bookshop. Lots of stuff, comics, books, dvds, collectibles, etc. I strolled into it when going back to the airport, that's why I had only around 5 minutes to look around. But the shop is definitely worth visiting. They offer English books among Swedish-lang ones, with 50:50 proportions in quantity. Science Fiction Bokhandeln

The darkest and best supplied metal shop I've ever seen - thou haven't seen many of such stores. Again, stumbled on it on my way to the airport, so had only around 5 minutes to look around, scan through t-shirts, etc :D Sound Pollution

Some t-shirt shop, had also lots of underwear, dark costumes, etc. I think there may be more to it than I noticed.

Unfortunately hadn't found this during my wandering. Will try next time: Roberta Settels shoes designs

Don't know the name of this packed with designer items place, but it was warm and cosy :) Prices were high and the utensils neat. Bore marks of "Swedish design school".