Integrating Amazon S3 with Spring Boot
Many times, we as developers need to deal with large objects such as images, photos, documents, and so on. There are different alternatives available to store these objects: as a binary in the database, or directly on the server file system. But these options are usually quite inefficient, so as a general rule, it will always be better to choose a cloud storage.
Amazon Simple Storage Service (Amazon S3) is a popular cloud storage that offers industry-leading scalability, data availability, security, and performance.
In this article, we are going to explore how to integrate S3 into your Spring Boot Kotlin applications.
Configuring Credentials
This tutorial starts with the assumption that you have already created an S3 Bucket on AWS. If you haven’t, please see how to create your first S3 bucket.
First, add the AWS Java SDK For Amazon S3 dependency to your build.gradle file.
implementation(“com.amazonaws:aws-java-sdk-s3:${version}”)
Set up your development environment with the Access Key, the Secret Access Key and the Bucket Name. In this example we are setting up the Access Keys and the Bucket into three environment variables: S3_ACCESS, S3_SECRET and S3_BUCKET.
Now, let’s add the properties into your application.yml as shown below.
s3:
access: ${S3_ACCESS:}
secret: ${S3_SECRET:}
bucket: ${S3_BUCKET:}
region: us-east-1
Create a ConfigurationProperties data class to read the properties under the prefix "s3" defined in the application.yml.
@ConstructorBinding
@ConfigurationProperties(prefix = "s3", ignoreUnknownFields = false)
data class S3Prop(
val access: String,
val secret: String,
val region: String,
val bucket: String
)
The S3Prop class should have a field with the value of each property.
Creating the Client
In a configuration class, we create a Bean instance of the AmazonS3 class. To do this, use the AmazonS3ClientBuilder class from the AWS SDK.
@Configuration
@EnableConfigurationProperties(S3Prop::class)
class S3Config(private val properties: S3Prop) {
@Bean
fun amazonS3(): AmazonS3 {
val credentials = BasicAWSCredentials(
properties.access,
properties.secret
)
return AmazonS3ClientBuilder.standard()
.withRegion(properties.region)
.withCredentials(
AWSStaticCredentialsProvider(credentials))
.build()
}
}
Here, the amazonS3 Bean is able to authenticate and make requests to Amazon Web Services. With the S3Prop properties, we can set the credentials (Access Key and Secret Key) in a BasicAWSCredentials instance.
We have also specified the Region chosen for the Bucket.
Now, let’s create a component class (the client) with three public methods:
Recommended by LinkedIn
@Component
class S3Client(
private val amazonS3: AmazonS3,
private val properties: S3Prop
) {
fun uploadObject(multipartFile: MultipartFile): URL {
val id = UUID.randomUUID().toString()
this.upload(id, multipartFile)
return amazonS3.getUrl(properties.bucket, id)
}
fun deleteObject(id: String) =
try {
amazonS3.deleteObject(
DeleteObjectRequest(properties.bucket, id)
)
} catch (e: Exception) {
throw RuntimeException(e)
}
fun modifyObject(id: String, multipartFile: MultipartFile): URL {
if (amazonS3.doesObjectExist(properties.bucket, id)) {
this.deleteObject(id)
}
this.upload(id, multipartFile)
return amazonS3.getUrl(properties.bucket, id)
}
private fun upload(id: String, multipartFile: MultipartFile) =
try {
val file = this.toFile(multipartFile)
amazonS3.putObject(
PutObjectRequest(properties.bucket, id, file)
.withCannedAcl(
CannedAccessControlList.PublicRead
)
)
file.delete()
} catch (e: Exception) {
throw RuntimeException(e)
}
private fun toFile(multipartFile: MultipartFile): File =
try {
val file = File(multipartFile.originalFilename!!)
FileOutputStream(file).use {
val fos = FileOutputStream(file)
fos.write(multipartFile.bytes)
fos.close()
return file
}
} catch (e: IOException) {
throw RuntimeException(e)
}
}
Upload Method
In the S3Client class, the uploadObject() method receives a MultipartFile object as argument and returns a URL instance with the resource location on Amazon S3.
In the example, we declare an Object Name (the object identifier in the bucket) with a random UUID value.
In the private method upload(), the putObject() method from the AWS SDK is used to upload files. This method requires a File instance as a parameter, so we will make the necessary conversion in the toFile() method.
Note that in the putObjectRequest instance, we add the CannedAccessControlList.PublicRead parameter into the withCannedAcl() method. This parameter sets the access permission to the object as “Public”, which means that anyone with the location URL will be able to access it.
Delete Method
The deleteObject() method receives the object identifier and removes the object from S3. Note that if we try to delete an object that does not exist, Amazon S3 will return a success message instead of an error message.
Modify Method
The modifyObject() method receives the object identifier and the MultipartFile object, and returns the location URL. There is no such thing as a modify method in the AWS SDK, so we first delete the object — if it exists — and then upload the new one with the same object identifier.
Creating the Endpoints
In order to test what we have done so far, we create three endpoints to create, delete and modify objects within the S3 Bucket:
@RestController
@RequestMapping("/s3/files")
class S3Controller(private val client: S3Client) {
@PostMapping(headers = ["content-type=multipart/*"])
fun upload(
@RequestPart(value = "file") file: MultipartFile
): ResponseEntity<String> {
val url = client.uploadObject(file)
val location = url.toURI()
return ResponseEntity
.created(location)
.build()
}
@DeleteMapping("/{id}")
@ResponseStatus(HttpStatus.NO_CONTENT)
fun delete(
@PathVariable id: String
) = client.deleteObject(id)
@PutMapping("/{id}", headers = ["content-type=multipart/*"])
fun modify(
@PathVariable id: String,
@RequestPart(value = "file") file: MultipartFile
): ResponseEntity<String> {
val url = client.modifyObject(id, file)
val location = url.toURI()
return ResponseEntity
.status(HttpStatus.NO_CONTENT)
.location(location)
.build()
}
}
Here in the POST and PUT methods, we accept requests with Content-Type=Multipart/* and we return the location of the object on Amazon S3 in the Location header of the response.
Let’s try to upload an object using curl.
$ curl -L -v POST ‘http://localhost:8080/s3/files’ \
--form ‘file=@"/Users/jonathan/Desktop/Amazon-S3-Logo.png"’
> POST /s3/files HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.79.1
> Accept: */*
> Content-Length: 9465
> Content-Type: multipart/form-data; boundary=------------------------76b042a08cdad0f3
>
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 201
< Location: https://mybucket.s3.amazonaws.com/d655f16d-4981-487b-aa5f-9c14fed8412b
< Content-Length: 0
< Date: Fri, 07 Oct 2022 14:14:56 GMT
<
* Connection to host localhost left intact
As you can see, the Location header contains the URL with the file we just uploaded. You will receive a URL with the following structure:
https://<your-bucket>.s3.amazonaws.com/<object-identifier>
Thanks for reading. I hope this was helpful!
The example code is available on GitHub.