Introduction

So, you want to learn more about Step Functions and their application. Let me share a problem that has been stuck in my mind since the beginning of my career, and by solving this problem lets learn how Step Functions can be useful.

I am sure you have worked as an employee before and it always felt boring to go through the process of getting the products required for working-- such as a laptop, mouse, keyboard etc. Plus, handing over the products and maintaining the records is difficult for HR as well. And I always wanted a simple solution; so here I am with one using Step Functions.

Requirements

An Employee may require a few things from the Management team to get settled with their basic needs (i.e. laptop, keyboard/mouse etc.) And while this process may seem boring, in the backend there are several different process going on to keep count of the products and to check the voucher/points necessary to get the products. By the time the management team hands over the items to the employee it will seem like ages have gone by.
‍

Let's break down the problem:

• Employee wants accessories from the company, which is required for work which may be a keyboard, upgraded laptop, mouse etc.
• Employee makes a request to management.
  • Management validates whether the employee has the required points / voucher provided from the company to get the requested product.
  • Management should have the product in their inventory.
• Once the validation is done, a request email is sent to HR, and the process is on-hold until HR approves the request.
  • Once HR approves the request:
    • An update is made to the product table about the count of the product.
    • An update of points on the Employee has been deducted.
    • A "success" notification is sent to the Employee.
  • If HR declines the Request:
    • A "failure" notification is sent to the Employee with the valid reason.

Solution

Is it hard to follow the bullet points? No worries, here is a flow level diagram which explains the process that AWS services use to tackle this problem:

The image above shows the Lambda, DDB, SNS in a flow-- seems weird, right? Of course not! This is the AWS Service named: Step Functions.

What is a step function? Step Functions are based on state machines and tasks. In Step Functions, state machines are called "workflows", which are a series of event-driven steps. Each step in a workflow is called a "state".

Step Functions have 2 types of flow, an "Express" and "Standard" flow. To know more go through the doc: Step Function Flow.

Step Functions are also known as an "Orchestration Service", which directs different AWS services in the sequence. Step Functions not only orchestrate, but also perform the state, add a human in step, parallel run of a different service, map/loop through the data, and retry for the failure task.

Here is the list of things you can do with a Step Function:

Lets understand the different applications of the states in a Step Function. Using these states we can approach our problem more effectively.

• Choice State: Choice state is where we can have an array of choice rules to make our decision. When writing choice rules we use comparison operators.
• Map State: Map state receives an array of JSON data and can do up to 40 concurrent iterations.
• Parallel State: Parallel state is the place where you can run multiple things in parallel. Each parallel execution is known as branches.
• Wait State: Wait state keeps the state machine on hold for a specified amount of time, and delays the execution until given approval.
• Success State: Indicates the success of the execution.

Implementation of the Solution

Since the workflow shared above in the Solution section only gives the mind map of the process, lets get our hands dirty by implementing the Step Function.

It should look something like this:

The state machine workflow seems to be complex. Lets simplify the approach by breaking down the steps. Each step has input by using it to make some changes and when successful, it'll send an output.

Step 1: The first task that the Step Function is executing; it fetches the employee and product info.

Lambda will have the permission to fetch the employee and product info from there respective table.

Employee Info:

{
    id: "employee#14",
    name: "employee14",
    joinedDate: "20-10-2019",
    points: 500,
    voucherId: "EMPOFTHEMONTH#APRIL#2024"
}

Product Info:

{
    id: "product#12",
    name: "Mac Air M1",
    count: 22
}

By the end, Lambda will return the combined info as the output from the Lambda which should look like:

{
    step: "getEmpProdInfo",
    employee:{
        id: "employee#14",
        name: "employee14",
        joinedDate: "20-10-2019",
        points: 500,
        voucherId: "EMPOFTHEMONTH#APRIL#2024"
    },
    product: [
      {
        id: "product#12",
        name: "Mac Air M1",
        count: 22
      },
      {
        id: "product#1",
        name: "MKeyboard",
        count: 2
      }
   ],

}

‍So once you have a Lambda there will be 200% chance that this execution might fail, and once the failure happens in the Lambda, then the Step Function's execution will return a failure message to the end user. Now the question arises, how can we solve this issue? To handle the failure message the Step Function provides try the "re-try" method: when the execution fails, you can always re-try 3/5 times to get a successful response.

To add re-try to the Lambda, navigate to the error-handling section in the console of the state description,  enter the error-type for which Lambda should re-try, give it a number of max retries, and decide on the wait period in-between the re-tries.

In the code level, as well, we can specify the retry policy for the state, as shown below:

"Retry": [
        {
          "ErrorEquals": [
            "Lambda.ServiceException",
            "Lambda.AWSLambdaException",
            "Lambda.SdkClientException",
            "Lambda.TooManyRequestsException"
          ],
          "IntervalSeconds": 1,
          "MaxAttempts": 3,
          "BackoffRate": 2
        }
    ]

‍Step 2: Now that we have the employee & product info, we now need to make a decision on the provided data. To make a decision, lets use the choice state, where we define the choice rules to match the condition on which a choice is made whether to approve / reject the employee request.

// pusedo code to decide whether an employee is eligible for the approval of the request

(employee.points === valid || employee.voucher !== null && product.count > 0)
  wait for the HR response and continue the next process.

‍


Step 3: Once the choice is made and the employee is eligible for the requested product, an approval is given, then before sending the product and email to the employee, we need to update the DDB table of the given product and employee info. Since we have multiple requested products, we can make use of the Map state. To make changes to all the product info at the same time we can make use of the Parallel state.

Step 4: Once the update on the DDB is done, we need to send the notification to the employee. The notification is sent in 2 scenarios, one is when the approval is given and another if a request has been rejected. To do that as soon as the step above is completed, using the SNS, a notification is sent.

And by the end, execution is terminated and a response is sent to the end user.

Conclusion

In summary, the idea is to use Step Functions, its types, and different states. By using everything in this article, you will have a deeper understanding of utilizing different states for different scenarios, and how Step Function can handle the different AWS services like Lambda, DDB, and SNS.

References

https://docs.aws.amazon.com/step-functions/latest/dg/choosing-workflow-type.html