From the course: Data Engineering on AWS: Data Cataloging, Processing, Analytics, and Visualization

Unlock this course with a free trial

Join today to access over 25,500 courses taught by industry experts.

Solution: Read catalog data using Athena

Solution: Read catalog data using Athena - Amazon Web Services (AWS) Tutorial

From the course: Data Engineering on AWS: Data Cataloging, Processing, Analytics, and Visualization

Start my 1-month free trial Buy for my team

Solution: Read catalog data using Athena

“

(bright music) - [Instructor] In order to query this data via Athena, first of all we need to create a Glue crawler. So let's go to Glue Console here and into the Crawlers section. Let's create a new crawler, and we can name it sensorCrawler. As we know that we are going to crawl the sensor's data. And then you can add a data source. Our data is in S3, and let's pick the bucket from here. Either we can choose all the folders in this bucket or we can go to precise folder as well. And then we are going ahead creating a new IAM role. Here we need to choose the database, the analyticsdb in our case, and we want to run this crawler on demand. So now this crawler is in READY state, let's run this crawler. So what this crawler will do? It will create a schema for the data available in our S3 bucket. And once we have the schema available means once we have done the cataloging of this data, then we can query it like any…

Contents