Example usage for com.amazonaws.services.s3 AmazonS3 selectObjectContent

Introduction

In this page you can find the example usage for com.amazonaws.services.s3 AmazonS3 selectObjectContent.

Prototype

SelectObjectContentResult selectObjectContent(SelectObjectContentRequest selectRequest)
        throws AmazonServiceException, SdkClientException;

Source Link

Document

This operation filters the contents of an Amazon S3 object based on a simple Structured Query Language (SQL) statement.

Usage

From source file:org.apache.flink.streaming.tests.util.s3.S3QueryUtil.java

License:Apache License

/** Run SQL query over non-compressed CSV file saved in s3 object. */
static String queryFile(AmazonS3 s3client, String bucket, String s3file,
        @SuppressWarnings("SameParameterValue") String query) {
    SelectObjectContentRequest request = generateBaseCSVRequest(bucket, s3file, query);
    final AtomicBoolean isResultComplete = new AtomicBoolean(false);
    String res;//from w ww.j a v a  2  s.c o  m
    try (SelectObjectContentResult result = s3client.selectObjectContent(request);
            SelectObjectContentEventStream payload = result.getPayload();
            ByteArrayOutputStream out = new ByteArrayOutputStream()) {
        InputStream resultInputStream = payload.getRecordsInputStream(new SelectObjectContentEventVisitor() {
            @Override
            public void visit(SelectObjectContentEvent.EndEvent event) {
                isResultComplete.set(true);
            }
        });
        copy(resultInputStream, out);
        res = out.toString().trim();
    } catch (Throwable e) {
        System.out.println("SQL query failure");
        throw new RuntimeException("SQL query failure", e);
    }
    /*
     * The End Event indicates all matching records have been transmitted.
     * If the End Event is not received, the results may be incomplete.
     */
    if (!isResultComplete.get()) {
        throw new RuntimeException("S3 Select request was incomplete as End Event was not received.");
    }
    return res;
}

---