From the course: Data Engineering on AWS: Data Cataloging, Processing, Analytics, and Visualization
Join today to access over 25,500 courses taught by industry experts.
Serverless processing with Lambda: Demo - Amazon Web Services (AWS) Tutorial
From the course: Data Engineering on AWS: Data Cataloging, Processing, Analytics, and Visualization
Serverless processing with Lambda: Demo
- [Instructor] Now this is the time to process the ingested data. We can use different processing methods to transform data and it completely depends on the use case and the tech stack being used. Here I'm covering most used processing techniques in big data world. So first of all, let's implement a very common use case that the moment data file is coming to S3 bucket we need to upload that data into DynamoDB table. We are going to implement this as a serverless application means we will not provision any EC2 instances for this. For this setup, we will need following, that first of all we need to set up IAM policy and rules so it has access on the required services. Then we will need S3 bucket where incoming file will be landed. Then we will create a sample CSV data file and we'll also create a DynamoDB table to accommodate this data. Next, we need to write this logic somewhere. So we will write this in Lambda to create it…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.