AWS Step Function Parameters

To solve a particular business problem I needed to find a solution to pass input parameters into a step function and pass a value from one lambda function to another, via that same step function.

The article image shows the process and will assist in the understanding of the solution.

To put into context, a trigger on an s3 bucket initiates a call to a lambda function (lam1), which then executes the step function (step1) The step function invokes a second lambda function (lam2). On successful completion of lam2, the step function then invokes a third lambda (lam3).

From the s3 event context received at lam1, I extracted just the key, also referred to as the prefix. I then populated a dictionary with this value.

source_key = event['Records'][0]['s3']['object']['key']

input_dict = {
    input_dict['source_key'] = source_key
}        

In otherwords, the source_key variable contains the prefix (or path) of the file that was just uploaded to the s3 bucket.

Then, I initiated the execution of step1 from lam1 by using boto3

sfn_client = boto3.client('stepfunctions')
       
response = sfn_client.start_execution(
      stateMachineArn = step_function_arn,
      input = str(input_dict).replace("'","\"")
)        

To utilise this information within step1 I needed to add

"InputPath": "$",
"ResultPath":"$"        

to each state within step1, the data being assigned to InputPath.

Then, as part of the lam2 state definition within step1, I specified the following payload to pass to lam2

"Parameters":{
    "FunctionName": "arn:aws:lambda:eu-central-1:account:function:lam2", 
    "Payload":{         
          "source_key.$":"$.source_key"  
     }   
}        

In order for this to work, we need to place .$ after the parameter name (source_key) and then specify $. to extract the source_key value from InputPath

Then, within lam2, I captured the input parameter value from the event context

source_key = event['source_key']        

At the end of lam2 I added the following

return {'curated_key_suffix': result}        

This returned a value back to step1 which was stored in

 "ResultPath":"$",        

Then, provided that lam2 succeeded, I could invoke lam3 with the following payload

"Parameters":{
    "FunctionName": "arn:aws:lambda:eu-central-1:account:function:lam3", 
    "Payload":{         
          "curated_key_suffix.$":"$.Payload.curated_key_suffix"  
     }   
}         

lam3 received the parameter data as an event context

curated_key_suffix = json.dumps(event['curated_key_suffix']).strip('"')