This post acts as an introduction to the microservice landscape. It covers common design patterns that are useful when building production grade microservice architectures.

Defining a Microservice

Microservices are a flexible software development technique that helps to implement applications as a collection of loosely coupled services. Since each service is independent from one another, they can be scaled and released indepedently. Each service acts as a single business function, so the smaller size tends to make them easier to manage and debug. Additionally, increased isolation lends itself to better security.

In order for each microservice to act as an autonomous component, it must fulfill the following criteria:

  • Shared-nothing architecture: it doesn’t share data in databases with other microservices
  • Separate runtime processes: it runs in a separate runtime process (e.g., a Docker container)
  • Well-defined interfaces: messages are sent to other microservices using APIs and message formats that are stable, well-documented, and conform to a defined versioning strategy
  • Stateless: incoming requests can be handled by any of a microservice’s instances

So how big should a microservice be? As a general rule of thumb, a microservice should be small enough to fit in the head of a developer, but big enough not to jeopardize performance and/or data consistency.

For more information on containers, container orchestration, and service meshes, see my previous post which goes into more depth on these topics.

Microservice Challenges

Now that we’ve taken a look at the potential benefits we can harvest by using microservices, we must also consider the following issues that can arise from autonomous components:

  • Keeping configuration up-to-date for many small components can be challenging
  • Synchronous communication can cause cascading failures, especially under high load
  • Performing root cause analysis by tracking a request that involves many components is difficult
  • Analyzing usage of hardware resources across components can be challenging
  • Manual management of many components becomes costly and error-prone

We must also consider the fact that decomposing an application into a group of autonomous components forms a distributed system, which are notoriously difficult to work with. In the early 1990s, Peter Deutsch famously came up with the following 8 fallacies of distributed computing:

  1. The network is reliable
  2. Latency is zero
  3. Bandwidth is infinite
  4. The network is secure
  5. Topology doesn’t change
  6. There is one administrator
  7. Transport cost is zero
  8. The network is homogeneous

Peter Deutsch stated that all these assumptions are commonly made when initially building distributed systems, but they all prove false in the long run. Building microservices based on these false assumptions leads to solutions that are prone to both problems that occur in other microservice instances and network interruptions. Assuming that there is always something going wrong in the system landscape is a good approach to designing microservice architectures.

Design Patterns

In order to help mitigate challenges with microservices, we will now look at a selection of common design patterns. This is not intended to be exhaustive, but instead should be considered a minimal set of design patterns required to handle the previously mentioned challenges. The following table outlines a number of design patterns alongside the problem that they arise from and the solution that they provide.

Design Pattern Problem Solution
Service Discovery How can clients find microservices and their instances? Microservice instances are typically assigned dynamically allocated IP addresses when they start up, which makes it difficult for a client to find out who it needs to make a request to. A new service discovery service can be added to keep track of currently available microservice instances and their corresponding IP addresses.
Edge Server Often times in a system landscape, we want to expose some microservices to the outside world while hiding others from external access. Additionally, any exposed services must be protected against malicious clients. Add a new edge server to the system landscape to handle all incoming requests. This edge server will behave similarly to a reverse proxy and can also perform dynamic load balancing by integrating with the discovery service.
Reactive Microservices Synchronous communication is implemented using blocking I/O, which means that a thread is allocated from the operating system for the length of the request. A rise in the number of concurrent requests has potential to result in running out of available threads in the operating system. This can result in problems ranging from crashing servers to longer response times. Instead, we can use non-blocking I/O to ensure that no threads are allocated while waiting for processing to occur in another service.
Central Configuration Applications are traditionally deployed together with their configuration settings. In a microservice architecture, it becomes challenging to get a complete picture of the configuration in place for running instances. Additionally, it becomes difficult to make sure microservice instances are updated correctly. We can add a new configuration server to the system landscape for storing configuration information for microservices.
Centralized Log Analysis An application traditionally writes log events on the local machine that the application runs on. When you have a large number of deployed microservice instances on a large number of smaller servers, it becomes challenging to get a view of the health of the system landscape if each microservice instance writes to its own log file. A new component can be added to the system landscape that is capable of managing centralized logging. This includes detecting new microservice instances for log collection, storing log events in a structured and searchable way, and providing APIs and/or graphical tools for querying/analyzing logs events.
Distributed Tracing It needs to be possible to track requests and messages that flow between microservices in the system. In order to track the processing that happens between cooperating microservices, we need to related requests and messages are marked with a common correlation ID that is part of log events. The centralized logging service can then be used to find all related log events.
Circuit Breaker Microservice architectures that use synchronous communication can expose themselves to a chain of failure. If one microservice stops responding, the clients that are waiting on responses can become non-responsive, which results in a problem that propagates recursively throughout the system landscape. A circuit breaker can be added that prevents new outgoing requests from a caller if it detects a problem with the service it calls.
Control Loop In a microservice architecture, it is very difficult to manually detect and correct problems such as crashed microservice instances. A control loop can be added to the system landscape that can constantly observe the actual state of the system landscape. It does this by comparing the actual state with the desired state, which is specified by the operators. Depending on this outcome, it can take appropriate action.
Centralized Monitoring and Alarms If usage of hardware resources and/or response times become unacceptably high, it can be challenging to discover the root cause. A monitoring service can be added to the system landscape to collect metrics like hardware resource usage for each microservice instance.

Software Solutions

In order to meet the previously mentioned challenges, we can use the following open source tools:

  • Spring Boot
  • Docker
  • Kubernetes
  • Istio

We can use Java and Spring Boot to write the microservices, but the same principles will apply across other programming languages and frameworks. It should be emphasized that we are building language agnostic microservices capable of being deployed in an actual production environment. As such, we can take advantage of Kubernetes for container orchestration and the Istio service mesh.

The following table maps each of the design patterns to the corresponding functionality in each tool that can be used:

Design Pattern Spring Boot Kubernetes Istio
Service Discovery   Kubernetes kube-proxy and service resources  
Edge Server   Kubernetes ingress controller Istio ingress gateway
Reactive Microservices Spring Reactor and Spring WebFlux    
Central Configuration   Kubernetes ConfigMaps and Secrets  
Centralized Log Analysis   Elasticsearch*, Fluentd*, and Kibana*  
Distributed Tracing     Jaeger
Circuit Breaker     Outlier detection
Control Loop   Kubernetes controller manager  
Centralized Monitoring and Alarms   Grafana* and Prometheus* Kiali, Grafana, and Prometheus

* = not technically part of Kubernetes, but is easily deployed together with Kubernetes

Summary

In this post, we defined what microservices are, what challenges they present, and looked at design patterns that can be used to combat these challenges. Additionally, we figured out which open source tools will be a good match for implementing these design patterns.