Add multiple Lambda event notifications using Terraform

Looking to have a simple main terraform file ( main.tf ) and get variables from multiple tfvars dynamically. This post gives you how to deploy multiple lambda bucket event configuration without hardcoding.

Module Structure

Let's assume below is the folder structure.

my_directory
   |---main.tf
   |---variables.tf
   |---tfvars_directory
            |---example_1.tfvars
            |---example_2.tfvars

variables.tf

Lets define all variables with default values in variables.tf file

variable "env"                  { default = "dev" }
variable "region"               {default = "ap-south-1" }
variable "bucket_name"          { default = "" }
variable "bucket_arn"           { default = "" }
variable "lambda_function_arn"  { default = "" }
variable "lambda_list"          { default = [] }
variable "prefix"               { default = "" }
variable "suffix"               { default = "" }

main.tf

provider "aws" {
  region                         = "${var.region}"
}


resource "aws_lambda_permission" "allow_bucket" {
  count                          = length(var.lambda_list)
  statement_id                   = "AllowExecutionFromS3Bucket_${count.index}"
  action                         = "lambda:InvokeFunction"
  function_name                  = "${lookup(var.lambda_list[count.index], "lambda_function_arn")}"
  principal                      = "s3.amazonaws.com"
  source_arn                     = "${lookup(var.lambda_list[count.index], "bucket_arn")}"
}

resource "aws_s3_bucket_notification" "bucket_notification"{
  bucket                         = "${var.bucket_name}"

  dynamic lambda_function {
    for_each                     = var.lambda_list
    content {
      lambda_function_arn        = lambda_function.value["lambda_function_arn"]
      events                     = ["s3:ObjectCreated:*"]
      filter_prefix              = lambda_function.value["prefix"]
      filter_suffix              = lambda_function.value["suffix"]
    }
  }

  depends_on                     = [aws_lambda_permission.allow_bucket]
}

tfvars

Now lets define our tfvars files, example_1.tfvars file has multiple lambdas defined under same s3 bucket where as example_2.tfvars has single lambda defined

example_1.tfvars

region                           = "ap-south-1"
environment                      = "dev"
bucket_name                      = "my_bucket1"
lambda_list = [
    {
        lambda_function_arn      = "arn:aws:lambda:ap-south-1:12345678901:function:my_lambda1"
        prefix                   = "folder1/sub_folder1/"
        suffix                   = ".csv"
        lambda_function_name     = "my_lambda1"

    },
    {
        prefix                   = "folder1/sub_folder2/"
        suffix                   = ".parquet"
        lambda_function_arn      = "arn:aws:lambda:ap-south-1:12345678901:function:my_lambda2"
        aws_lambda_function_name = "my_lambda2"
    }
]

example_2.tfvars file has single lambda defined

example_2.tfvars

region                           = "ap-south-1"
environment                      = "dev"
bucket_name                      = "my_bucket"
lambda_list = [
    {
        lambda_function_arn      = "arn:aws:lambda:ap-south-1:12345678901:function:my_lambda"
        prefix                   = "folder/sub_folder/"
        suffix                   = ".zip"
        lambda_function_name     = "my_lambda"

    }
]

Happy Coding !!!