本文档旨在指导开发者如何使用Spring Boot和AWS SDK从S3存储桶中读取包含多个JSON对象的文本文件，并将其转换为Java对象列表。文章将提供两种实现方法，一种是将S3文件读取到本地文件系统，另一种是在内存中直接处理数据，并提供详细的代码示例和配置说明。
在开始之前，请确保您已经具备以下条件：
首先，需要在pom.xml文件中添加AWS SDK for Java S3的依赖。
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>2.17.285</version>
</dependency>定义一个Java类来映射JSON对象。例如，如果S3中的文件包含以下JSON数据：
{
"name":"rohit",
"surname":"sharma"
}
{
"name":"virat",
"surname":"kohli"
}可以创建一个名为Person的类：
public class Person {
private String name;
private String surname;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getSurname() {
return surname;
}
public void setSurname(String surname) {
this.surname = surname;
}
@Override
public String toString() {
return "Person{" +
"name='" + name + '\'' +
", surname='" + surname + '\'' +
'}';
}
}创建一个配置类来初始化S3Client。
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Bean;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
@Configuration
public class AwsS3ClientConfig {
@Bean
public S3Client s3Client(){
AwsBasicCredentials awsBasicCredentials = AwsBasicCredentials.create("ACCESS_KEY_ID", "SECRET_ACCESS_KEY");
return S3Client
.builder()
.region(Region.US_EAST_1)
.credentialsProvider(StaticCredentialsProvider.create(awsBasicCredentials))
.build();
}
}注意: 将ACCESS_KEY_ID和SECRET_ACCESS_KEY替换为您的AWS凭证。 建议使用IAM角色或环境变量来管理凭证，而不是硬编码在代码中。 此外，Region.US_EAST_1也需要替换成你实际使用的AWS区域。
Easily find JSON paths within JSON objects using our intuitive Json Path Finder
30
以下代码演示了如何从S3读取文件，将其保存到本地文件系统，然后将文件内容转换为Person对象列表。
import org.springframework.stereotype.Service;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import org.springframework.beans.factory.annotation.Autowired;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
@Service
public class AwsS3Service {
private final S3Client s3Client;
@Autowired
public AwsS3Service(S3Client s3Client) {
this.s3Client = s3Client;
}
public List<Person> readFileAndCreateList(String bucketName, String keyName) throws IOException {
final Path file = readFile(bucketName, keyName);
return convertFileToList(file);
}
private Path readFile(String bucketName, String keyName) throws IOException {
GetObjectRequest getObjectRequest = GetObjectRequest
.builder()
.bucket(bucketName)
.key(keyName)
.build();
final byte[] bytes = s3Client
.getObject(getObjectRequest)
.readAllBytes();
final Path path = Paths.get("demo.txt");
Files.write(path, bytes);
return path;
}
private List<Person> convertFileToList(Path path) throws IOException {
final List<String> lines = Files.readAllLines(path);
StringBuilder json = new StringBuilder();
List<Person> persons=new ArrayList<>();
for (String line : lines) {
if ("{".equals(line)) {
json = new StringBuilder("{");
} else if ("}".equals(line)) {
json.append("}");
persons.add(new ObjectMapper()
.readValue(json.toString(), Person.class));
} else {
json.append(line.trim());
}
}
return persons;
}
}以下代码演示了如何直接从S3读取文件内容到内存中，然后将文件内容转换为Person对象列表，避免了本地文件系统的IO操作。
import org.springframework.stereotype.Service;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import org.springframework.beans.factory.annotation.Autowired;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.io.ByteArrayOutputStream;
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.io.BufferedReader;
@Service
public class AwsS3Service {
private final S3Client s3Client;
@Autowired
public AwsS3Service(S3Client s3Client) {
this.s3Client = s3Client;
}
public List<Person> readFileAndCreateObjectList(String bucketName, String keyName) throws IOException {
final List<String> lines = readFile(bucketName, keyName);
return convertFileLinesToObjectList(lines);
}
private List<String> readFile(String bucketName, String keyName) throws IOException {
GetObjectRequest getObjectRequest = GetObjectRequest
.builder()
.bucket(bucketName)
.key(keyName)
.build();
byte[] bytes;
try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
s3Client
.getObject(getObjectRequest)
.transferTo(byteArrayOutputStream);
bytes = byteArrayOutputStream.toByteArray();
}
List<String> lines=new ArrayList<>();
try(ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
InputStreamReader inputStreamReader = new InputStreamReader(byteArrayInputStream);
BufferedReader bufferedReader=new BufferedReader(inputStreamReader)){
while (bufferedReader.ready()){
lines.add(bufferedReader.readLine());
}
}
return lines;
}
private List<Person> convertFileLinesToObjectList(List<String> lines) throws IOException {
StringBuilder json = new StringBuilder();
List<Person> persons = new ArrayList<>();
for (String line : lines) {
if ("{".equals(line)) {
json = new StringBuilder("{");
} else if ("}".equals(line)) {
json.append("}");
persons.add(new ObjectMapper()
.readValue(json.toString(), Person.class));
} else {
json.append(line.trim());
}
}
return persons;
}
}创建一个Spring Boot Application类，并注入AwsS3Service，然后调用相应的方法。
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.CommandLineRunner;
import org.springframework.beans.factory.annotation.Autowired;
import java.util.List;
@SpringBootApplication
public class DemoApplication implements CommandLineRunner {
private final AwsS3Service awsS3Service;
@Autowired
public DemoApplication(AwsS3Service awsS3Service) {
this.awsS3Service = awsS3Service;
}
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class);
}
@Override
public void run(String... args) throws Exception {
//KEY_NAME==filename.txt
final List<Person> peoples =
awsS3Service
.readFileAndCreateList("BUCKET_NAME", "KEY_NAME");
System.out.println(peoples);
}
}注意: 将BUCKET_NAME和KEY_NAME替换为您的S3存储桶名称和对象键名。
如果一切配置正确，运行Spring Boot应用程序，您将在控制台中看到从S3读取并转换为Person对象的列表。
[Person{name='rohit', surname='sharma'}, Person{name='virat', surname='kohli'}]本文档介绍了如何使用Spring Boot和AWS SDK从S3存储桶中读取JSON对象列表的两种方法。您可以根据实际需求选择适合的实现方式。希望本文档能够帮助您更好地理解和应用Spring Boot与AWS S3集成。
以上就是使用Spring Boot从AWS S3中读取JSON对象列表的详细内容，更多请关注php中文网其它相关文章！
每个人都需要一台速度更快、更稳定的 PC。随着时间的推移，垃圾文件、旧注册表数据和不必要的后台进程会占用资源并降低性能。幸运的是，许多工具可以让 Windows 保持平稳运行。
Copyright 2014-2025 https://www.php.cn/ All Rights Reserved | php.cn | 湘ICP备2023035733号
239
收藏
