13 May, 2015
I've been working with Vlad Mettler for a couple of years now on microservice architecture and implementation. In fact, we gave a talk last year about Hexagonal Microservices and the impact they have on organisational strucutre at the Microservices User Group at Skills Matter.
In this article I want to develop the theory further to introduce the concept of a component called a dispatcher and to introduce a different pattern of microservice design I'm calling "Circuit Microservices" that rely on fixed communication protocols but are simpler to implement and require a different testing strategy compared with the Hexagonal Microservices presented in the video linked above.
The key charactistics of microservices for me are that they:
tell don't ask pattern)For me one of the advantages of microservices is that you can easily draw diagrams that actually relate to the real world. This means those boxes that you use to communicate your designs actually make sense.
You'll see me drawing lots of diagrams like this:
User Dispatcher Mail
+=========+ +---------+ +=======
| | | | |
+=+ +=+ +-+ +-+ +=+
--> |> )| ))-->--) |) >| >>-----> |>
+=+ +=+ +-+ +-+ +=+
| | | | |
+=========+ +---------+ +=======
Unit Unit Unit Unit Unit
The boxes with md_code(====) at the top and bottom represent services of some kind, in this case a user registration service, and a mail delivery service. The line md_code(`))----)') represents an event being fired from the User service and listened to by the dispatcher. The line md_code(>>----->) represents an invocation of an API method on the Mail service, perhaps to have a user registration email sent.
The dents cut into the service represent outgoing and incoming ports. In this
case the User service's outgoing port simply notifies subscibed services of
events ensuring the service knows nothing about the outside world (e.g.
md_code(user.user_registered_event()')). The Mail service's *incoming* port simply handles invocation requests and returns responses (e.g. md_code(mail.send()')).
At this point we hit a problem. If the outgoing port of the user service is blindly sending events, and the incoming port of the Mail service is expecting API calls to be made to it, how do we connect the two together?
You'll see a number of solutions proposed in the microservices community:
Change the User service's port to make explict API calls to the Mail services incoming port rather than just sending out events
Change the Mail service's incoming port to be an event handler so that it can handle the events, rather than having an explict API that can be called
Leave the User service and Mail service as they are, but put a third component between them that listens to the User events and dispatches them to calls to the Mail service
Change both the User service and the Mail service. Then add a queue component between them so that the User service publishes messages to the queue, and the Mail service acts like a worker, taking messages off the queue and processing them
I'm not really a fan of 1. and 2. because both of these solutions couple together the operation of the mail and user services, making them less re-usable as standalone components. If the User service has to know how to call md_code(mail.send()) it is less re-usable. If the Mail service has to know how to handle md_code(user.user_registered_event), it is less re-usable.
That leaves us with 3. and 4. I think there are strong advantages to option 3:
The orchestration of which events trigger which actions is kept entirely separate from the work each service does, meaning that in different deployments, the services themselves stay the same, but only the dispatcher components change. The dispatcher component effectively contains all the business rules, in a neatly factored out place.
By moving the dispatcher component to be bolted on to either the User service or the Mail service, you automatically get options 1. or 2. anyway, so you have flexibility going forward
You don't need to build a separate dispatcher between every pair of services, you just build a generic one that takes different configuration of which actions to perform for which events, and use that wherever it is needed
You can choose any configuration of dispatchers. Perhpas you want one global dispatcher to be god over all your services, or maybe you want a different dispatcher for each pair of services. Most likely you'll have something in between.
Any time you want to experiment with new things, you simply re-configure the dispatcher to handle new events, or to do new things with the existing events, no changes are needed to the existing services.
The dispatcher provides a place where errors can be handled, without those errors ever needing to be propogated back to the components that fired the events and shouldn't be aware of what things external to them broke (since that would couple them together)
Option 4 provides similar opportunities, but has some immediate advantages over 3.:
Since it is a queue, it already handles failutres and re-tries which is handy
Since the consuming service is acting as a worker, it is a. never going to be overloaded b. is easy to scale by adding more workers
In practice of course, a dispatcher could be given similar characteristics. The queue option has disadvantage though:
Again, this could be seen as positiive in some ways, because all messages go through the queue, and direct access is not allowed, making logging and monitoring simpler.
I think the major reason why I'm currently preferring the dispatcher option to a queue option is that the dispatcher is just another microservice in its own right, but one with different protocol semantics from the others. Separating out the bits of code designed to be re-usable from those designed to glue together those parts is always a good idea. Here the microservices are re-usable and make no assumptions about the outside world, the dispatchers are business-domain specific and contain all the rules about what should happen under which circumstances.
The second reason is that queues already have their advocates, followers and worldview. Once you start worrying topics, exchanges and all the rest, you move further from thinking about your business goals and delivering value to the users to achieve them. Staying close to the user and the business goals is always a good thing.
This also provides quite a good separation of concerns. Once a re-usable dispatcher has been written, architecutre teams can work with product teams to focus on configuring dispatchers to deliver value to the customers and the business, and domain-specific engineering teams can focus in a specialised way on their particular microservice.
The nice thing about having everything triggered by publishers and subscribers to events is that if one service goes down, all the sending service has to do is start streaming all failed events to a local logfile. When the service comes back, it can read the logfile and start sending all those events again before sending new ones. It is a very simple, but very robust pattern. If events are sent in the same order, most things should be fine. They should be idemotant so sending the same event twice shouldn't be a problem.
Since dispatchers sit in the middle between components, they are also a logical place performing monitoring and handing the scaling of the other components. Since all they are doing is sending and receiving messages, whereas the microservices are doing the work, the dispatchers should never be a bottleneck in the system, (unless they aren't well designed of course).
So far I haven't mentioned a concept called adapters. In this theory, you can see there are four types of port:
+=+ +-+ +-+ +-+
)| )) ) |) >| >> > |>
+=+ +-+ +-+ +-+
Outgoing Incoming Outgoing Incoming
Event Event Command Command
The semantics of these ports are all different. In order to connect them up we need to translate a function call in an outgoing port to say an HTTP request which gets received at the other end and turned into a function call on the incoming port. The components that do this are called adapters.
In the diagrams above, you can think of these being the ports:
+=+ +-+ +-+ +-+
)| |) >| |>
+=+ +-+ +-+ +-+
Outgoing Incoming Outgoing Incoming
Event Event Command Command
Port Port Port Port
And these being the adapters:
)) ) >> >
Outgoing Incoming Outgoing Incoming
Event Event Command Command
Adapter Adapter Adapter Adapter
By keeping the adapters sepearate from the ports, it becomes possible to change the physical transport between microservices, without changing the microservice knowing. This can be handy if you change your deployment infrastructure and can no-longer rely on transports that you used to use. For example you might switch to Azure or AWS and need to integrate with their queue system instead of one of you own. If adapters are kept separate, this is easy to do.
Although writing adapters is a good idea, it can be harder than you expect because most frameworks try to make life easier for you by coupling together various adapters and just exposing an interface for you to handle the core logic. If you are able to make a decision that you are never going to change your communication protocol you might decide that the extra work of de-coupling a framework into clean adapters, or building your own strucutre including adapters is not worth the effort.
The other thing to bear in mind is that it is very hard to write adapters that adapt the semantics of the underlying protocol rather than just its interface. When you change protocols, it is likely a bit of re-work to the microservice will need to be made too. Another possibility is building the adapters at the first point you need them.
If you don't have a passing test, you don't know your code works now. But there is another more important reason for testing - to facilitate refactoring so that you still know that your code works when you are in the middle of changing it.
The problem is that good testing is hard and requires discipline. Too often people test the wrong things and/or test them in the wrong way. This makes testing expensive, and less valuable than it should be.
What you really want to do is to test everything that matters, and nothing that doesn't. Since every piece of your system matters, you might conclude that you should test every single piece of your code in the smallest chunks, and test every intereaction between pieces. That would be a lot of work though, and the tests would end up serving to prevent refactoring rather than facilitate it because they would be getting in the way of change. What you want is tests that test the boundary of code that is likely to change together. That way, your tests facilitate your work to refactor internals, and enforce the boundary interface with the outside world.
In the world of microservices, what is the code that is likely to change together? It is the individual services and dispatchers. So I'd avocate only testing that they behave correctly at their boundaries, and that they interact correctly together.
Here are two services with their ports symbolised by the square brackets, communicating with each other:
--- [ ] --- [ ] ---
If you used the ports and adatpers style mentioned in the last section, you can test everything with three tests - one unit test for each of the internals of the services, with all I/O mocked out:
--- >[ ]< --- [ ] ---
--- [ ] --- >[ ]< ---
and one communication test to ensure the communication happens correctly. The communication test happens from the inside of the outgoing port to the outside of the incoming port, and including both the adapters:
--- [ >] --- [< ] ---
In reality, the unit tests are very easy to write with whatever mock tool your language uses but the communication tests can be a bit tricky. One way forward is with a "programmable mock". Let's look at the top part of the interaction between the User and Dispatcher services:
User Dispatcher
=========+ +--------
| |
+-+ +=+ +=+ +-+
|D| | ) system under test ) | |F|
+++ +=+ +=+ +++
| |
+----------- Test Driver ----------+
Here, the driver on the inside of the User outgoing port, has a direct link to the responder on the far side of the Dispatcher incoming port so that it can pre-program the responses it is expects. This is handy, because it allows you to write communication tests in the same style you would write your unit tests, mocking out the responses from the other ports on each test. The programming of the fake on the far side could be via a protocol like HTTP.
Since each test is isolated from every other part of the system, each test can run very fast. You never need fixtures because you can mock responses from the far side of every port for every sort of test. If you have written your tests properly you don't need end-to-end tests because each part of the system has already been tested, as has every way each part of the system can communicate with every other part.
So far, so good. But there is a problem here. With the ports and adapters approach described so far, we are optimising for the ability to change the communication protocols between microservices at a point in the future. We can do this by changing the adapters, and we don't need to change anything else, all the tests are totally untouched, brilliant!
But what happens if we want to re-write a microservice in a different language in the future? Well, our adapters are useless, they are implemented in the wrong langauge. Our unit tests and communication tests also need to be thrown away, they are also in the wrong language. Even our ports are only of any use in that they might hint at a good API to implement in the new language. We've put a lot of effort into making things flexible, only to throw it away.
Might there have been a better approach?
Suppose for a moment that we were absolutely sure we were always going to use HTTP/1.1 as a transport between all microservices, as well as between a client (the browser) and a server. If we really could make this decision then it might make sense to test slightly differently. Instead of a ports and adatpers approach implemented as physical APIs in the language of the microservice, I can write tests directly in HTTP against the microservices.
Instead of the places to test looking like this:
--- >[ ]< --- [ ] ---
--- [ ] --- >[ ]< ---
--- [ >] --- [< ] ---
We can simply test HTTP like this:
>--- [ ] ---< [ ] ---
--- [ ] >--- [ ] ---<
One immediate advantage is that there are now only two sets of tests that need writing not three. A less clear advantage is that depending on how you implement your tests, and the language you use, you might find that you end up using a similar SDK for your tests as you do for your microservices, so the duplication of effort between tests and implementation is not so large (I'm not sure if this is such a good thing yet).
The key benefit though it this - I can now replace any microservice with any other and as long as the new one has the same HTTP interface, it will work. I can ensure it has the same HTTP interface by running the old tests, completely unchanged. The tests therefore serve to help in refactoring as well as to prove the behavior of the new service.
I call this the "circuit" microservices approach because it reminds me a lot of how electronig circuits work. You test electronic components individually with external current and voltages, and once you've proven they work you wire them all up in a circuit. It is the wires themeselves where you place your oscilliscope or multimeter to test, not the insides of the capacitor or the chip.
There is a lot more to say about all this, but I'm hopeful a lot of it will be covered in the upcoming O'Reilly book on microservices. You can get a free preview at nginx.com. That book is the reason I chose not to write a book about microservices myself with another publisher, so I hope it is good :)
Copyright James Gardner 1996-2020 All Rights Reserved. Admin.