Introduction

This blog post explores the benefits of using OpenAPI Specification (OAS) with Amazon API Gateway for developing RESTful APIs. We'll delve into the importance, the evolution and the necessity for a common API definition standard. We'll then provide a deep dive into how OAS and Amazon API Gateway work together seamlessly. Finally, we'll discuss the pros, cons, and areas of improvement.

The Importance of APIs in Our Era

source: https://developers.giphy.com/branch/master/static/api-512d36c09662682717108a38bbb5c57d.gif

In today's digital age, APIs play a crucial role in enabling software systems to interact seamlessly. They serve as the backbone for building integrations and microservices architectures, allowing disparate systems to communicate and share data efficiently. APIs have evolved significantly since their inception, moving from simple XML-based services to robust JSON-based formats like OpenAPI v3, which provides a standard way to describe RESTful APIs.

The need for OpenAPI arose due to the increasing complexity of APIs and the necessity for consistent documentation and tooling across various projects. With OpenAPI, developers can define APIs once and generate documentation, client SDKs, server stubs, and other artifacts automatically.

According to Postman’s 2023 State of the API Report, the API economy continues to grow, with businesses recognizing the value of APIs for innovation and competitive advantage. As stated in the report:

11% of respondents defined themselves as API-first leaders, up from 8% in each of the previous two years. This elite group excels in almost every metric. For instance, API-first leaders produce APIs faster and report fewer failures. And when an API fails, most API-first leaders can restore it in less than an hour—a feat that only a minority of respondents can match.

What is OpenAPI?

OpenAPI, formerly known as Swagger, is a vendor-neutral, language-agnostic specification that defines how HTTP-based APIs should be structured and documented. It is a community-driven effort that aims to provide a standard format for describing APIs, making them easier for both humans and computers to understand and use. It was originally based on the Swagger 2.0 Specification, donated by SmartBear Software in 2015. OpenAPI version 3 introduced several improvements over its predecessors, including support for asynchronous communication, improved support for machine-to-machine authentication, and a more expressive model for defining API paths and operations.

What is Design-first API development?

The term design-first for API development has become synonymous with OpenAPI but what does it mean? A design-first approach to API development prioritizes the planning and documentation of the API before writing any code. This method allows for a collaborative effort where stakeholders can contribute to the API design, ensuring that the API meets the desired specifications and business requirements.

On the other hand, a code-first approach to API development starts with the implementation of the API in code, and then generates a machine-readable API definition from that code. This can involve coding an API from business requirements and then creating an API description using comments or annotations, or even manually writing a description from scratch. The code-first approach does not necessarily exclude API design; instead, the design process is integrated within the code documents.

Comparing the two approaches:

Design-First:

• Better collaboration: Stakeholders participate in the design process, leading to a shared understanding.
• Consistency: A single API design serves as the contract for the API, ensuring consistency and interoperability.
• Reduced development costs: Issues are caught early in the design phase, which is cheaper to resolve than during implementation.

Code-First:

• Rapid development: Coding can be faster, particularly for simple APIs or when immediate deployment is necessary.
• Familiarity: Many developers are comfortable with this approach, as it follows the traditional software development process.
• Direct control: Developers have immediate access to the implementation, allowing for direct iteration and testing.
• Seamless integration: When building on an existing codebase, the API can be integrated naturally into the existing system.

Code-First Disadvantages:

• Documentation: Creating documentation can become an afterthought, leading to poor documentation practices.
• Higher costs: Changes to an already-coded API are more expensive than changes to an API design.

Though the code-first approach is more understood and popular among developer-centric API development initiatives, it doesn’t give room for other stakeholders like testers, product managers, and technical writers to understand or provide feedback before many hours are invested in coding the API. Also, documentation is an afterthought which is a well-known issue with software development. It becomes more expensive to make changes to already-coded APIs when other stakeholders suggest changes after the API has already been coded.

Choosing between a design-first and code-first approach depends on the project's requirements, the team's familiarity with the tools, and the level of collaboration needed. For complex projects requiring extensive collaboration and a high degree of standardization, the design-first approach is often preferred. For simpler projects or when quick deployment is a priority, the code-first approach may be more suitable.

OpenAPI document structure

OpenAPI Descriptions (OAD) are text documents written in YAML or JSON formats. It could be one or more documents that are linked by references to each other. A minimal OAD must contain some fields that adhere to the structure defined in the OpenAPI Specification. The full structure is extremely long so we will stick with the minimal OAD to explain some of the important fields.

  
openapi: 3.0.0
info:
	# Info component contains general information about the API
  title: Your API Title
  description: Brief description of your API
  version: 1.0.0
servers:
  # At least one server is required
  - url: https://api.example.com/v1
paths:
  # At least one path is required
  /example:
    get:
		# at least one operation for a path is required
      summary: Get Example
      description: Endpoint to get example data
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string
                    example: "Hello, World!"
  

This example includes:

• 'openapi': Version of the OpenAPI Specification
• 'info': Basic information about your API, including title and version
• 'paths': Defines at least one API endpoint (path) and associated operations (e.g., GET)
• 'get': Specific details about the GET operation for the defined path
• 'responses': Expected response codes and descriptions
• 'content': Information about the response data format
• 'schema': Definition of the response data structure
• ': At least one server where your API is hosted

Remember, this is just a minimal functional template. You need to further define reusable components (like schemas, parameters, and responses), add more paths and operations, include API security modes (such as OAuth Flows Object, and Security Scheme Object), and  Specification Extensions (which we will use in Amazon API Gateway). You can use the online Swagger editor example to have an idea of what a full API definition looks like.

Amazon API Gateway

Amazon API Gateway is a fully managed service that makes it easy to create, publish, maintain, monitor, and secure REST, HTTP, and WebSocket APIs at any scale. It acts as the front door for your web-based applications and can transform inbound web requests into events that are processed by Lambda functions, AWS Services, and HTTP endpoints. It supports the OpenAPI specification, allowing developers to import and export OpenAPI definitions easily.

API Gateway offers two types of RESTful API products: REST APIs and HTTP APIs.

• API Gateway REST API: REST APIs are the traditional API Gateway offering, providing a full range of features for building RESTful APIs, including advanced authorization, caching, and request/response transformations.
• API Gateway HTTP API: HTTP APIs are designed for building low-latency, cost-effective APIs that are well-suited for serverless architectures. They offer a streamlined experience with built-in support for CORS, JWT authorizers, and automatic deployment.

At the time of this post, the API Gateway HTTP API doesn’t support OpenAPI Schemas, which are crucial in avoiding code duplication. If you import an OpenAPI file into an API Gateway HTTP API, the schemas, request validation, and requestBody fields will be ignored but the API Gateway REST API supports them. The OpenAPI schema data model will be imported as API Gateway Models which are the same thing. So note that further explanations of the API Gateway Extensions for OpenAPI are based on the API Gateway REST API offering.

Furthermore, API Gateway HTTP APIs only support Lambda proxy and HTTP proxy integrations. The Lambda proxy integration is a lightweight API Gateway integration with AWS Lambda service which allows you to integrate an API method or the whole API with a Lambda function. With this integration type, API Gateway sends the request directly from the client to the Lambda function without any modifications which is useful for rapid prototyping or lightweight APIs.

API Gateway REST APIs also support the Lambda Proxy integration type but to benefit from all of the features of API Gateway services, it also supports a non-Lambda proxy or custom integration type. With this integration type, API Gateway can modify the request before sending it to the Lambda function and also modify the response from the same function before sending it to the client. You have more options and can be used to provide feature-rich APIs but it is time-consuming to set up properly and you need to know VTL language.

How does Amazon API Gateway support OpenAPI?

Amazon API Gateway can import and use OpenAPI specifications to define REST APIs. This means you can leverage the benefits of OpenAPI's design-first approach while utilizing API Gateway's powerful features and integrations with other AWS Services. However, AWS has a lot of vendor-specific features and functionalities not covered by the base standard. While OAS is a powerful and comprehensive specification, it aims to provide a universal and neutral way to describe APIs across different platforms. This means it cannot encompass all the unique features and functionalities offered by individual cloud providers.

OpenAPI solves the problem of platform-specific features by allowing the use of Extensions. With Extensions, additional data can be added by third parties to extend the specification at certain points.

Extensions are supported at the root level of the API spec and in the following places:

• 'info' section
• 'paths' section, individual paths, and operations
• operation parameters
• 'responses'
• 'tags'
• security schemes

The extension’s properties are implemented as patterned fields that are always prefixed by '"x-"'.

Amazon API Gateway Extensions to OpenAPI

Amazon API Gateway extensions start with '"x-amazon-apigateway-"'.

The extension value can be a primitive, an array, an object, or 'null'. If the value is an object or array of objects, the object’s property names do not need to start with 'x-'.

At the time of this post, there are over 29 API Gateway extensions to OpenAPI. A cool tip to know what API Gateway extensions are required is to first build your REST API from the API Gateway Console, then export it to an OpenAPI definition file.

Let's consider the example of the Hair Extensions API and explore the extensions added to the OpenAPI spec to support integration with Lambda functions using the API Gateway Lambda Proxy Integration:

  
...
paths:
  /extensions/{extensionId}:
    put:
      operationId: "updateExtensions"
      parameters:
      - name: "extensionId"
        in: "path"
        required: true
        schema:
          type: "string"
      requestBody:
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/extension"
        required: true
      responses:
        "200":
          description: "200 response"
          content: {}
        "400":
          description: "400 response"
          content: {}
      security:
				- ExtensionsAuthorizer: []
      - api_key: []
      x-amazon-apigateway-request-validator: "Validate body, query string parameters,\
        \ and headers"
      x-amazon-apigateway-integration:
        type: "aws_proxy"
        httpMethod: "POST"
        credentials:
          Fn::GetAtt: [LambdaExecutionRole, Arn]
        uri: 
          Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/arn:aws:lambda:us-west-1:404148889442:function:updateExtension/invocations"
        responses:
          default:
            statusCode: "200"
        passthroughBehavior: "when_no_match"
        contentHandling: "CONVERT_TO_TEXT"
...
components:
    securitySchemes:
        ExtensionsAuthorizer:
            type: apiKey
            name: Authorization
            in: header
            x-amazon-apigateway-authtype: cognito_user_pools
            x-amazon-apigateway-authorizer:
                type: cognito_user_pools
                providerARNs:
                  - Fn::GetAtt: [ExtensionsUserPool, Arn]
  

• x-amazon-apigateway-request-validator: This extension specifies the request validator to be used for validating incoming requests. It ensures that the request body, query string parameters, and headers conform to the specified schema (found in the components section not shown in the code snippet) before being forwarded to the backend Lambda function.
• x-amazon-apigateway-integration: This extension is an extended property of the OpenAPI Operation object. API Gateway uses it to define the integration settings for backend services, in this case, the API Gateway Lambda Proxy integration settings for a backend Lambda function. It includes configurations such as the type of integration (e.g. aws_proxy), the URI of the Lambda function, and response handling options.
• x-amazon-apigateway-authtype: This extension is used to define an Amazon Cognito Authorizer for the endpoint. We defined the extension in the components section so that it can be used by multiple endpoints.
• x-amazon-apigateway-authorizer: This extension defines an Amazon Cognito user pool to be applied for authorization of method invocations in API Gateway. We defined the extension in the components section so that it can be used by multiple endpoints.

When building Serverless APIs with API Gateway and Lambda, the 'x-amazon-apigateway-integration' extension must be added to the OpenAPI definition. There are other extensions that you might probably use. For example, the x-amazon-apigateway-authorizer object can be used to define a Lambda authorizer, Amazon Cognito user pool, or JWT authorizer like OAuth. Once more, you can refer to the list of all API Gateway extensions to OpenAPI.

Workflow for Integrating an OpenAPI Spec with Amazon API Gateway

The advantage of having your API defined as a text file like OpenAPI is that it can be part of the software development lifecycle and benefit from all the associated advantages. But correctly setting up the OpenAPI file for Amazon API Gateway with the necessary extensions might be tricky so some people first use the API Gateway Console to define their APIs, then export them as OpenAPI files. That way, they can see exactly which OpenAPI objects and API Gateway extensions are required. You can also define your API using an editor like Swagger.io or Postman and then import it into API Gateway as part of your workflow.

To integrate an OpenAPI spec with Amazon API Gateway using AWS SAM (Serverless Application Model), you can follow these steps:

1. Define the OpenAPI spec for your API, including endpoints, request/response schemas, and security definitions. This can be done with an OpenAPI Editor in your IDE or online with Swagger.io.
2. Add API Gateway extensions to the OpenAPI spec as needed to support your backend such as AWS Lambda. You can reference the API Gateway Extensions documentation or use the API Gateway Console to help export the updated version of the OpenAPI file with the necessary API Gateway extensions.
3. Use AWS SAM to define the API Gateway resource that references the OpenAPI spec file and connects the endpoints to the backend such as a Lambda function.

Below is a snippet on how you can define an API Gateway resource in your AWS SAM template file and reference your openAPI spec file.

  
...
Resources:
  HairExtensionsAPI:
    Type: AWS::Serverless::Api
    Properties:
      StageName: dev
      OpenApiVersion: 3.0.3
      DefinitionBody: # an OpenApi definition
        "Fn::Transform":
          Name: "AWS::Include"
          Parameters:
            Location: "hair-extension-api.yaml"
      EndpointConfiguration:
        Type: REGIONAL

...
  

    • 'OpenApiVersion': 3.0.3: This specifies the version of the OpenAPI specification being used.

    • 'DefinitionBody': This is where the OpenAPI definition for the API is specified. However, instead of directly providing the OpenAPI definition inline, it's using an AWS CloudFormation intrinsic function called 'Fn::Transform' to include the definition from an external file.

         •  'Name: "AWS::Include"': This specifies the transformation to apply to the included file. In this case, it's using 'AWS::Include' to include the contents of another file.

         •  'Parameters': This section contains parameters for the transformation.

              •  'Location: "hair-extension-api.yaml"': This specifies the location of the file containing the OpenAPI definition. It's expected to be named "hair-extension-api.yaml".

      4. Deploy the SAM template to create the API Gateway REST API or HTTP API along with the associated Lambda functions by running the following commands from the root of your project:

  
sam build && sam deploy --guided
  

Finally, you can either use the AWS CLI or have a look at the API Gateway Console to see the resources that were created. The code snippets above are found in this GitHub repo.

For CDK lovers, you can have a look at this two-part blog post by Serverless Advocate.

Pros and Cons of Using OpenAPI with Amazon API Gateway

Pros

• Standardization: OpenAPI provides a standardized way to describe APIs, promoting consistency and interoperability across different services and platforms.
• Design-First Approach: By adopting a design-first approach with OpenAPI, developers can clearly define API contracts before implementing backend logic, fostering better collaboration and reducing development time.
• Lambda Proxy Integration: The integration of OpenAPI with Amazon API Gateway's Lambda Proxy Integration simplifies the process of routing requests to backend Lambda functions, reducing overhead and latency.

Cons

• Limited Compatibility: Not all features supported by Amazon API Gateway are directly compatible with the OpenAPI specification, requiring the use of extensions or manual configuration.
• Learning Curve: Working with OpenAPI and Amazon API Gateway may have a learning curve for developers unfamiliar with these technologies, especially when dealing with complex integrations and configurations. Furthermore, large OpenAPI documents are difficult to maintain without dedicated tools.

Room for Improvement

While OpenAPI and Amazon API Gateway offer powerful capabilities for building and deploying APIs, there are areas where improvements could enhance the development workflow and user experience:

• Documentation and Tutorials: Providing comprehensive documentation and tutorials can help developers navigate the complexities of working with OpenAPI and Amazon API Gateway more effectively.
• Simplified Integration: Streamlining the process of integrating OpenAPI with API Gateway, particularly for advanced features, could make it more accessible to developers and reduce the need for manual configuration.
• Enhanced Compatibility: Continuously improving compatibility between OpenAPI and Amazon API Gateway, such as adding support for importing OpenAPI models in HTTP APIs, would simplify the development process and reduce friction.

Conclusion

In conclusion, leveraging OpenAPI with Amazon API Gateway offers a powerful combination for building scalable and robust APIs in the cloud. By adopting a design-first approach with OpenAPI and harnessing the features of API Gateway, developers can build robust, scalable, and well-documented APIs that meet the demands of today's dynamic and distributed software environments. While there are challenges and areas for improvement, the benefits of using OpenAPI with Amazon API Gateway will make the future of interoperable vendor-neutral APIs a reality.

References

https://www.postman.com/state-of-api/

https://learn.openapis.org/specification/structure

https://swagger.io/specification/

https://spec.openapis.org/oas/v3.1.0

https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-rest-api.html

https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api.html

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions.html

https://swagger.io/docs/specification/openapi-extensions/

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions.html