In this short post, I want to demonstrate an approach to decouple a developed system from external messages that it should handle. Our goal is to fulfill functional requirements and keep the developed system flexible enough to add new features as effortlessly as possible.

The project

Incoming messages will contain a resource id, some flags, and a list of checks to perform against that resource.

%{
  resource_id: String.t(),
  checks: [type:  String.t()],
  flags: [String.t()]
}

Message typespec

Those checks can be costly, and some messages will carry the "validation" flag to give us a hint when we can skip them. Should we encounter it in a message, we will evaluate the worthwhileness of running each check.
If a message doesn't have it - it's assumed to be worthy of processing.

Validation flag also alters output behavior - if it's enabled, we are always obligated to publish a result. If not, and all checks failed - it is considered useless, and we don't need to publish anything in response to it.

The first pitfall: coupling with an upstream data source

First off, we want to be in "partnership" relations with upstream data service.
That means we don't want to be dependent on the decisions that they make and the sure way to do it is to decouple our code from the shape of the structure that they control, and we don't

Assume that message that we ingest and the message that we will produce are two different beasts.

We are not going to extend the original message with our fields. Instead, we are going to create a new one.

%Apple{
  resource_id: String.t(),
  actions: [type:  String.t()],
  flags: [String.t()]
}

Incoming message typespec

%AppleJuice {
  resource_id: resource_id,
  results:  [%{result: String.t()}]
}

Results message typespec

The names are contrived on purpose, but I hope they will illustrate the relationship though.

The second pitfall: drawing border in a wrong place

Here is how we will process an incoming message (%Apple{}) - we will take checks from, iterate through them one by one, process, gather results, and put them into an outbound message structure (%AppleJuice{}).

defmodule Juicer do
  def process(message) do
    results = Checks.run(message.resource_id, message.checks, message.flags)

    %Result{
       resource_id: resource_id,
       results: results
     }
  end
end

Message processing entry point

Next - the code that will support that behaviour:

defmodule Checks do
  alias Check

  def process(resource_id, checks, flags) do
    Enum.map(checks, &  perform_check(resource_id, &1, flags))
  end

  defp perform_check(resource_id, check, flags) do
    check
    |> wrap(hostname, flags)
    |> validate()
    |> process()
    |> unwrap()
  end

  defp wrap(check, hostname, flags) do
    Check.new(check, hostname, flags)
  end

  defp validate(check) do
    if valid?(check) do
      Check.validated(check, valid?(Check.resource_id(check))
    else
      Check.validated(check, true)
    end
  end

  defp valid?(resource_id) do
    ExpensiveService.valid_resource?(resource_id)
  end

  defp process_action(%{validated: false} = check) do
    action
  end

  defp process_action(%{validated: true} = check) do
      Check.result(check, ProcessinService.do_magic(check.resource_id))
  end

   defp unwrap(action) do
      if Check.value_changed?(check) or Check.discovery?(check) do
      Check.unwrap(action)
    else
      nil
    end
  end
end

Message processing logic

defmodule Check do
  @enforce_keys [:resource_id, :flags, :source,]
  defstruct validated: nil, resource_id: nil, flags: result: nil, source: nil

  def new(source, resource_id, flags) do
    %__MODULE__{resource_id: resource_id, flags: flags, source: source}
  end

  def discovery?(check) do
    "validate" in check.flags
  end

  def validated(check, status) do
    %{check | validated: status}
  end

  def resource_id(check) do
    check.resource_id 
  end

  def result(check, result) do
    %{check | result: result}
  end

  def unwrap(check) do
    %{results: check.result}
  end

  def value_changed?(action) do
    check.result != Map.get(check.source, :cached_value)
  end
end

Check "value object"

Now we are completely protected from any changes to the data structure!
But isn't there something irritating with all those functions inside Check module and frivolous treating of the structure itself - sometimes, we accessed its properties through functions and sometimes directly?

Indeed we pushed the border way too far south!

The SRP principle tells us that module should have only one reason to change. And fortunately, we already established that reason in the first section: change of the shape of the data structure shouldn't cause us to change our code.
The Check struct here serves as a miniature "anti-corruption layer," and we will draw the border between fields that we control in it and those that we don't!

The only functions that should stay in Check are those that access the data source field - the data that we don't have any power over.

defmodule Check do
  @enforce_keys [:resource_id, :flags, :source,]
  defstruct validated: nil, resource_id: nil, flags: result: nil, source: nil

  def new(source, resource_id, flags) do
    %__MODULE__{resource_id: resource_id, flags: flags, source: source}
  end

  def unwrap(check) do
    %{results: check.result}
  end

  def value_changed?(action) do
    check.result != Map.get(check.source, :cached_value)
  end
end

Check "value object"

defmodule Checks do
  alias Check

  def process(resource_id, checks, flags) do
    Enum.map(checks, &perform_check(resource_id, &1, flags))
  end

  defp perform_check(resource_id, check, flags) do
    check
    |> wrap(hostname, flags)
    |> validate()
    |> process()
    |> unwrap()
  end

  defp wrap(check, hostname, flags) do
    Check.new(check, hostname, flags)
  end

  defp validate(check) do
    if valid?(check) do
      %{check | validated: valid?(check.resource_id)}
    else
      %{check | validated: true}
    end
  end

  defp process_action(%{validated: false} = check) do
    action
  end

  defp process_action(%{validated: true} = check) do
    %{check | result: ProcessinService.do_magic(check.resource_id)}
  end

  defp unwrap(action) do
    if Check.value_changed?(check) or discovery?(check) do
      Check.unwrap(action)
    else
      nil
    end
  end

  def discovery?(check) do
    "validate" in check.flags
  end

  defp valid?(resource_id) do
    ExpensiveService.valid_resource?(resource_id)
  end
end

Message processing logic

Notice less mouthful and how much more pleasant to read it become!

We also kept the new and unwrap functions in the Check module. Creating Factory and Formatter doesn't make any sense in our case, and they can stay there for a while :)

Things to consider

Think of the following new requirements:

publish results to a queue communicated to us in %Apple{}:

%Apple{
  resource_id: String.t(),
  actions: [type: String.t()],
  flags: [String.t()],
  output: String.t()
}

Incoming message typespec

add a validated property to action's result if an incoming message has to be validated

@type result :: %{result: String.t() } | %{result: String.t(), validated: boolean() }
%AppleJuice {
  resource_id: resource_id,
  results:  [result]
}

Results message typespec

Will the structure that we laid down help you accommodate them, or will it get in your way?

Summary

"Good fences make good neighbors" because borders separate what you control from what you don't control.

Now you have one more heuristic to apply SRP!

Resources

Here are couple of links if you want to learn more about modularization:

On the Criteria To Be Used in Decomposing Systems into Modules

"Value Object" and "Anticorruption Layer" sections in "DDD Quickly" (free ebook)

← Yevhenii Kurtov's blog

On drawing borders