How to create AWS Transfer for SFTP Custom Identity Provider for Active Directory

Sep 29, 2019


Volodymyr Rudyi

Active Directory often used in corporate world to authenticate and authorize users on a big scale. SFTP is another protocol popular for data exchange, integration and ETL(Extract-Transform-Load) processes. Integrating both of them can be very beneficial for the organization. Let's see how it can be done using AWS.

What is Active Directory?

Developed by Microsoft, Active Directory is a service that stores information about various network resources and also, where applicable, maps them to physical network addresses. Data in the Active Directory is represented as a set of objects with attributes and relations between them. Those relations can be represented using the tree data structure:

Active Directory Example

To access the Active Directory the LDAP protocol is used. LDAP(Lightweight Directory Access Protocol) is an open-source protocol supported by many platforms and languages.

The Idea

We are going to use AWS Transfer for SFTP with a custom authentication configured to allow uploading files to S3 via SFTP using Active Directory credentials:

Active Directory and SFTP Diagram

We have the following steps involved:

  1. Client initiates an SFTP transfer
  2. AWS Transfer for SFTP configured to use a custom identity provider, sends a request to the AWS API Gateway
  3. AWS API Gateway invokes our custom AWS Lambda function
  4. AWS Lambda verifies authentication information against the AWS Directory
  5. If authentication information is correct data is being transferred from/to S3.

Everything, except for the step (4) is a standard behavior provided and managed by AWS. We only need to implement the authentication itself.

It looks rather simple, but there is a catch: AWS Directory Service can be used only inside a VPC. So if our Lambda needs to validate credentials against Active Directory, it also should be inside a VPC.

Let's see how we can describe such infrastructure.


In our company, even for demo projects we try to use infrastructure-as-a-code. This way it's easier both to share the code and delete unneeded resources. We use tools like AWS CloudFormation, Serverless or Terraform. In this tutorial, I'll go with Serverless + CloudFormation becase it's easier to work with AWS Lambda this way.

Let's start with the VPC:

     Type: AWS::EC2::VPC
       CidrBlock: ""
       EnableDnsHostnames: true
       EnableDnsSupport: true

We are also going to create two subnets:

     Type: AWS::EC2::Subnet
         Ref: vpc
           - 0
           - Fn::GetAZs: ""
     Type: AWS::EC2::Subnet
         Ref: vpc
           - 1
           - Fn::GetAZs: ""

As you can see, we don't provide Availability Zone (AZ) names. Instead, we use a dynamic list of available AZs, since they depend on a region.

Now it's time to declare our AD itself:

     Type: AWS::DirectoryService::SimpleAD
       Password: ${env:AD_ADMIN_PASSWORD}
       Size: "Small"
           - Ref: subnet1
           - Ref: subnet2
           Ref: vpc

There is a variable "ad_password" which can be set using environment variable when the "sls deploy" command is being executed. It will be the password for the AD Administrator user.

Here is also a definition of the AWS Lambda Function in the serverless.yml:

   handler: handler.authorize
   role: lambdaExecutionRole
       - Ref: authorizerSecurityGroup
       - Ref: subnet1
       - Ref: subnet2
     - http:
         path: /servers/{serverId}/users/{user}/config
         method: GET
         authorizer: aws_iam
     SFTP_USER_ROLE_ARN: !GetAtt sftpUserRole.Arn
         - ":"
         - - "arn"
           - Ref: "AWS::Partition"
           - "s3::"
           - ${env:TRANSFER_BUCKET_NAME}
     LDAP_DIRECTORY_NAME: sftpdemo

You may be wondering, why I'm using non-existing domain) as a LDAP server hostname. We can do it, since I'm going to create a Route 53 Private Hosted Zone inside our VPC. It will also have a record for our Active Directory IPs so we can access it using the hostname:

     Type: AWS::Route53::HostedZone
       Name: ""
         - VPCId:
             Ref: vpc
           VPCRegion: !Ref "AWS::Region"
     Type: "AWS::Route53::RecordSet"
         Ref: privateHostedZone
       Type: A
       TTL: "900"
       ResourceRecords: !GetAtt activeDirectory.DnsIpAddresses
     DependsOn: privateHostedZone

The next step is to create AWS Transfer for SFTP server. You can read our blog post about it. Six months later, CloudFormation(and thus Serverless) has a support for the AWS Transfer Server resource.

     Type: AWS::Transfer::Server
           Fn::GetAtt: ["transferInvocationRole", "Arn"]
             - ""
             - - https://
               - Ref: ApiGatewayRestApi
               - .execute-api.
               - Ref: "AWS::Region"
       IdentityProviderType: "API_GATEWAY"
         Fn::GetAtt: ["transferLoggingRole", "Arn"]

To provision the infrastructure, execute the following command:

sls deploy

Now we have all the infrastructure required and can proceed with our custom AWS Lambda function code.

Active Directory Authentication in AWS Lambda

To perform the actual authentication, we are going to use the same approach as in our previous blog post about AWS Transfer for SFTP custom identity providers. The only difference is that we are going to perform an LDAP bind instead of AWS Cognito authenticateUser API call. Here is the related code fragment:

exports.handler = async function(event) {
 return new Promise((resolve, reject) => {
   const username = event.pathParameters.user;
   const password = event.headers.Password;

   console.log(`Performing authentication for user ${username}`);

   client.bind(`${username}@${process.env.LDAP_DIRECTORY_NAME}`, password, err => {
     if (err) {
     } else {
       const response = {
         headers: {
           "Access-Control-Allow-Origin": "*",
           "Content-Type": "application/json"
         body: getSftpPolicy(username),
         statusCode: 200

A very important part here is the fact the bind call requires to pass the DN in the format <ldap-directory-netbios-name>@<username>. In the opposite case, the bind call will fail.


Flexibility of AWS Transfer for SFTP custom identity providers allows implementing authentication and authorization using virtually any data source. In a case if the organization uses Active Directory, it's possible to provide access to AWS S3 via SFTP to Active Directory users with a fine-grained level of access control.