List of usage examples for com.amazonaws.services.s3.model S3ObjectSummary getKey
public String getKey()
From source file:S3ClientSideEncryptionWithSymmetricMasterKey.java
License:Apache License
private static void deleteBucketAndAllContents(AmazonS3 client) { System.out.println("Deleting S3 bucket: " + bucketName); ObjectListing objectListing = client.listObjects(bucketName); while (true) { for (Iterator<?> iterator = objectListing.getObjectSummaries().iterator(); iterator.hasNext();) { S3ObjectSummary objectSummary = (S3ObjectSummary) iterator.next(); client.deleteObject(bucketName, objectSummary.getKey()); }//from w w w. j a v a 2 s . c o m if (objectListing.isTruncated()) { objectListing = client.listNextBatchOfObjects(objectListing); } else { break; } } ; client.deleteBucket(bucketName); }
From source file:BackupManager.java
License:Open Source License
public static void main(String[] args) throws IOException { AmazonS3 s3 = new AmazonS3Client( new PropertiesCredentials(BackupManager.class.getResourceAsStream("AwsCredentials.properties"))); String bucketName = "my-first-s3-bucket-" + UUID.randomUUID(); String key = "MyObjectKey"; System.out.println("==========================================="); System.out.println("Getting Started with Amazon S3"); System.out.println("===========================================\n"); try {/*from w ww .j av a2 s .com*/ /* * Create a new S3 bucket - Amazon S3 bucket names are globally unique, * so once a bucket name has been taken by any user, you can't create * another bucket with that same name. * * You can optionally specify a location for your bucket if you want to * keep your data closer to your applications or users. */ System.out.println("Creating bucket " + bucketName + "\n"); s3.createBucket(bucketName); /* * List the buckets in your account */ System.out.println("Listing buckets"); for (Bucket bucket : s3.listBuckets()) { System.out.println(" - " + bucket.getName()); } System.out.println(); /* * Upload an object to your bucket - You can easily upload a file to * S3, or upload directly an InputStream if you know the length of * the data in the stream. You can also specify your own metadata * when uploading to S3, which allows you set a variety of options * like content-type and content-encoding, plus additional metadata * specific to your applications. */ System.out.println("Uploading a new object to S3 from a file\n"); s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile())); /* * Download an object - When you download an object, you get all of * the object's metadata and a stream from which to read the contents. * It's important to read the contents of the stream as quickly as * possibly since the data is streamed directly from Amazon S3 and your * network connection will remain open until you read all the data or * close the input stream. * * GetObjectRequest also supports several other options, including * conditional downloading of objects based on modification times, * ETags, and selectively downloading a range of an object. */ System.out.println("Downloading an object"); S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); System.out.println("Content-Type: " + object.getObjectMetadata().getContentType()); displayTextInputStream(object.getObjectContent()); /* * List objects in your bucket by prefix - There are many options for * listing the objects in your bucket. Keep in mind that buckets with * many objects might truncate their results when listing their objects, * so be sure to check if the returned object listing is truncated, and * use the AmazonS3.listNextBatchOfObjects(...) operation to retrieve * additional results. */ System.out.println("Listing objects"); ObjectListing objectListing = s3 .listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix("My")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println(); /* * Delete an object - Unless versioning has been turned on for your bucket, * there is no way to undelete an object, so use caution when deleting objects. */ System.out.println("Deleting an object\n"); s3.deleteObject(bucketName, key); /* * Delete a bucket - A bucket must be completely empty before it can be * deleted, so remember to delete any objects from your buckets before * you try to delete them. */ System.out.println("Deleting bucket " + bucketName + "\n"); s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
From source file:S3Sample.java
License:Open Source License
public static void main(String[] args) throws IOException { /*/*from w w w .j a va 2 s . c o m*/ * Important: Be sure to fill in your AWS access credentials in the * AwsCredentials.properties file before you try to run this * sample. * http://aws.amazon.com/security-credentials */ AmazonS3 s3 = new AmazonS3Client( new PropertiesCredentials(S3Sample.class.getResourceAsStream("AwsCredentials.properties"))); String bucketName = "my-first-s3-bucket-" + UUID.randomUUID(); String key = "MyObjectKey"; System.out.println("==========================================="); System.out.println("Getting Started with Amazon S3"); System.out.println("===========================================\n"); try { /* * Create a new S3 bucket - Amazon S3 bucket names are globally unique, * so once a bucket name has been taken by any user, you can't create * another bucket with that same name. * * You can optionally specify a location for your bucket if you want to * keep your data closer to your applications or users. */ System.out.println("Creating bucket " + bucketName + "\n"); s3.createBucket(bucketName); /* * List the buckets in your account */ System.out.println("Listing buckets"); for (Bucket bucket : s3.listBuckets()) { System.out.println(" - " + bucket.getName()); } System.out.println(); /* * Upload an object to your bucket - You can easily upload a file to * S3, or upload directly an InputStream if you know the length of * the data in the stream. You can also specify your own metadata * when uploading to S3, which allows you set a variety of options * like content-type and content-encoding, plus additional metadata * specific to your applications. */ System.out.println("Uploading a new object to S3 from a file\n"); s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile())); /* * Download an object - When you download an object, you get all of * the object's metadata and a stream from which to read the contents. * It's important to read the contents of the stream as quickly as * possibly since the data is streamed directly from Amazon S3 and your * network connection will remain open until you read all the data or * close the input stream. * * GetObjectRequest also supports several other options, including * conditional downloading of objects based on modification times, * ETags, and selectively downloading a range of an object. */ System.out.println("Downloading an object"); S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); System.out.println("Content-Type: " + object.getObjectMetadata().getContentType()); displayTextInputStream(object.getObjectContent()); /* * List objects in your bucket by prefix - There are many options for * listing the objects in your bucket. Keep in mind that buckets with * many objects might truncate their results when listing their objects, * so be sure to check if the returned object listing is truncated, and * use the AmazonS3.listNextBatchOfObjects(...) operation to retrieve * additional results. */ System.out.println("Listing objects"); ObjectListing objectListing = s3 .listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix("My")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println(); /* * Delete an object - Unless versioning has been turned on for your bucket, * there is no way to undelete an object, so use caution when deleting objects. */ System.out.println("Deleting an object\n"); s3.deleteObject(bucketName, key); /* * Delete a bucket - A bucket must be completely empty before it can be * deleted, so remember to delete any objects from your buckets before * you try to delete them. */ System.out.println("Deleting bucket " + bucketName + "\n"); s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
From source file:S3ClientSideEncryptionAsymmetricMasterKey.java
License:Apache License
private static void deleteBucketAndAllContents(AmazonS3 client) { System.out.println("Deleting S3 bucket: " + bucketName); ObjectListing objectListing = client.listObjects(bucketName); while (true) { for (Iterator<?> iterator = objectListing.getObjectSummaries().iterator(); iterator.hasNext();) { S3ObjectSummary objectSummary = (S3ObjectSummary) iterator.next(); client.deleteObject(bucketName, objectSummary.getKey()); }//w ww .jav a 2 s . c o m if (objectListing.isTruncated()) { objectListing = client.listNextBatchOfObjects(objectListing); } else { break; } } client.deleteBucket(bucketName); }
From source file:AmazonS3Handler.java
License:Open Source License
public void listObject() { try {//from w w w. jav a2s . com // System.out.println("Creating bucket " + bucketName + "\n"); // s3.createBucket(bucketName); // System.out.println("Listing buckets"); // for (Bucket bucket : s3.listBuckets()) { // System.out.println(" - " + bucket.getName()); // } // System.out.println(); // System.out.println("Uploading a new object to S3 from a file\n"); // s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile()).withCannedAcl(CannedAccessControlList.PublicRead)); // System.out.println("Downloading an object"); // S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); // System.out.println("Content-Type: " + object.getObjectMetadata().getContentType()); // displayTextInputStream(object.getObjectContent()); System.out.println("Listing objects"); ObjectListing objectListing = s3 .listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix("My")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println(); // System.out.println("Deleting an object\n"); // s3.deleteObject(bucketName, key); // System.out.println("Deleting bucket " + bucketName + "\n"); // s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
From source file:alluxio.underfs.s3a.S3AUnderFileSystem.java
License:Apache License
/** * Lists the files in the given path, the paths will be their logical names and not contain the * folder suffix. Note that, the list results are unsorted. * * @param path the key to list/*from w ww . ja v a 2 s . c o m*/ * @param recursive if true will list children directories as well * @return an array of the file and folder names in this directory * @throws IOException if an I/O error occurs */ private String[] listInternal(String path, boolean recursive) throws IOException { path = stripPrefixIfPresent(path); path = PathUtils.normalizePath(path, PATH_SEPARATOR); path = path.equals(PATH_SEPARATOR) ? "" : path; String delimiter = recursive ? "" : PATH_SEPARATOR; Set<String> children = new HashSet<>(); try { ListObjectsV2Request request = new ListObjectsV2Request().withBucketName(mBucketName).withPrefix(path) .withDelimiter(delimiter).withMaxKeys(LISTING_LENGTH); ListObjectsV2Result result = null; while (result == null || result.isTruncated()) { // Query S3 for the next batch of objects result = mClient.listObjectsV2(request); // Advance the request continuation token to the next set of objects request.setContinuationToken(result.getNextContinuationToken()); // Directories in S3 UFS can be possibly encoded in two different ways: // (1) as file objects with FOLDER_SUFFIX for directories created through Alluxio or // (2) as "common prefixes" of other files objects for directories not created through // Alluxio // // Case (1) (and file objects) is accounted for by iterating over chunk.getObjects() while // case (2) is accounted for by iterating over chunk.getCommonPrefixes(). // // An example, with prefix="ufs" and delimiter="/" and LISTING_LENGTH=5 // - objects.key = ufs/, child = // - objects.key = ufs/dir1_$folder$, child = dir1 // - objects.key = ufs/file, child = file // - commonPrefix = ufs/dir1/, child = dir1 // - commonPrefix = ufs/dir2/, child = dir2 // Handle case (1) for (S3ObjectSummary obj : result.getObjectSummaries()) { // Remove parent portion of the key String child = getChildName(obj.getKey(), path); // Prune the special folder suffix child = CommonUtils.stripSuffixIfPresent(child, FOLDER_SUFFIX); // Only add if the path is not empty (removes results equal to the path) if (!child.isEmpty()) { children.add(child); } } // Handle case (2) for (String commonPrefix : result.getCommonPrefixes()) { // Remove parent portion of the key String child = getChildName(commonPrefix, path); // Remove any portion after the last path delimiter int childNameIndex = child.lastIndexOf(PATH_SEPARATOR); child = childNameIndex != -1 ? child.substring(0, childNameIndex) : child; if (!child.isEmpty() && !children.contains(child)) { // This directory has not been created through Alluxio. mkdirsInternal(commonPrefix); children.add(child); } } } return children.toArray(new String[children.size()]); } catch (AmazonClientException e) { LOG.error("Failed to list path {}", path, e); return null; } }
From source file:aws.example.s3.DeleteBucket.java
License:Open Source License
public static void main(String[] args) { final String USAGE = "\n" + "To run this example, supply the name of an S3 bucket\n" + "\n" + "Ex: DeleteBucket <bucketname>\n"; if (args.length < 1) { System.out.println(USAGE); System.exit(1);/*from w w w .j a va 2s .c o m*/ } String bucket_name = args[0]; System.out.println("Deleting S3 bucket: " + bucket_name); final AmazonS3 s3 = new AmazonS3Client(); try { System.out.println(" - removing objects from bucket"); ObjectListing object_listing = s3.listObjects(bucket_name); while (true) { for (Iterator<?> iterator = object_listing.getObjectSummaries().iterator(); iterator.hasNext();) { S3ObjectSummary summary = (S3ObjectSummary) iterator.next(); s3.deleteObject(bucket_name, summary.getKey()); } // more object_listing to retrieve? if (object_listing.isTruncated()) { object_listing = s3.listNextBatchOfObjects(object_listing); } else { break; } } ; System.out.println(" - removing versions from bucket"); VersionListing version_listing = s3.listVersions(new ListVersionsRequest().withBucketName(bucket_name)); while (true) { for (Iterator<?> iterator = version_listing.getVersionSummaries().iterator(); iterator.hasNext();) { S3VersionSummary vs = (S3VersionSummary) iterator.next(); s3.deleteVersion(bucket_name, vs.getKey(), vs.getVersionId()); } if (version_listing.isTruncated()) { version_listing = s3.listNextBatchOfVersions(version_listing); } else { break; } } System.out.println(" OK, bucket ready to delete!"); s3.deleteBucket(bucket_name); } catch (AmazonServiceException e) { System.err.println(e.getErrorMessage()); System.exit(1); } System.out.println("Done!"); }
From source file:aws.example.s3.ListObjects.java
License:Open Source License
public static void main(String[] args) { final String USAGE = "\n" + "To run this example, supply the name of a bucket to list!\n" + "\n" + "Ex: ListObjects <bucket-name>\n"; if (args.length < 1) { System.out.println(USAGE); System.exit(1);//from w w w .j ava 2s .c o m } String bucket_name = args[0]; System.out.format("Objects in S3 bucket %s:\n", bucket_name); final AmazonS3 s3 = new AmazonS3Client(); ObjectListing ol = s3.listObjects(bucket_name); List<S3ObjectSummary> objects = ol.getObjectSummaries(); for (S3ObjectSummary os : objects) { System.out.println("* " + os.getKey()); } }
From source file:aws.sample.S3Sample.java
License:Open Source License
public static void main(String[] args) throws IOException { /*/*from ww w .ja v a 2 s. c om*/ * Important: Be sure to fill in your AWS access credentials in the AwsCredentials.properties file before you try to run this sample. http://aws.amazon.com/security-credentials */ AmazonS3 s3 = new AmazonS3Client( new PropertiesCredentials(S3Sample.class.getResourceAsStream("/AwsCredentials.properties"))); String bucketName = "my-first-s3-bucket-" + UUID.randomUUID(); String key = "MyObjectKey"; System.out.println("==========================================="); System.out.println("Getting Started with Amazon S3"); System.out.println("===========================================\n"); try { /* * Create a new S3 bucket - Amazon S3 bucket names are globally unique, so once a bucket name has been taken by any user, you can't create another bucket with that same name. * * You can optionally specify a location for your bucket if you want to keep your data closer to your applications or users. */ System.out.println("Creating bucket " + bucketName + "\n"); s3.createBucket(bucketName); /* * List the buckets in your account */ System.out.println("Listing buckets"); for (Bucket bucket : s3.listBuckets()) { System.out.println(" - " + bucket.getName()); } System.out.println(); /* * Upload an object to your bucket - You can easily upload a file to S3, or upload directly an InputStream if you know the length of the data in the stream. You can also specify your own metadata when uploading to S3, which allows you set a variety of options like content-type and content-encoding, plus additional metadata specific to your applications. */ System.out.println("Uploading a new object to S3 from a file\n"); s3.putObject(new PutObjectRequest(bucketName, key, createSampleFile())); /* * Download an object - When you download an object, you get all of the object's metadata and a stream from which to read the contents. It's important to read the contents of the stream as quickly as possibly since the data is streamed directly from Amazon S3 and your network connection will remain open until you read all the data or close the input stream. * * GetObjectRequest also supports several other options, including conditional downloading of objects based on modification times, ETags, and selectively downloading a range of an object. */ System.out.println("Downloading an object"); S3Object object = s3.getObject(new GetObjectRequest(bucketName, key)); System.out.println("Content-Type: " + object.getObjectMetadata().getContentType()); displayTextInputStream(object.getObjectContent()); /* * List objects in your bucket by prefix - There are many options for listing the objects in your bucket. Keep in mind that buckets with many objects might truncate their results when listing their objects, so be sure to check if the returned object listing is truncated, and use the AmazonS3.listNextBatchOfObjects(...) operation to retrieve additional results. */ System.out.println("Listing objects"); ObjectListing objectListing = s3 .listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix("My")); for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println( " - " + objectSummary.getKey() + " " + "(size = " + objectSummary.getSize() + ")"); } System.out.println(); /* * Delete an object - Unless versioning has been turned on for your bucket, there is no way to undelete an object, so use caution when deleting objects. */ System.out.println("Deleting an object\n"); s3.deleteObject(bucketName, key); /* * Delete a bucket - A bucket must be completely empty before it can be deleted, so remember to delete any objects from your buckets before you try to delete them. */ System.out.println("Deleting bucket " + bucketName + "\n"); s3.deleteBucket(bucketName); } catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which means your request made it " + "to Amazon S3, but was rejected with an error response for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means the client encountered " + "a serious internal problem while trying to communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
From source file:awslabs.lab21.SolutionCode.java
License:Open Source License
@Override public void listObjects(AmazonS3 s3Client, String bucketName) { // Construct a ListObjectsRequest object using the provided bucket name. ListObjectsRequest listObjectsRequest = new ListObjectsRequest().withBucketName(bucketName); // Submit the request using the listObjects method of the s3Client object. ObjectListing objectListing = s3Client.listObjects(listObjectsRequest); // Take the results and write the object keys and sizes to the console. for (S3ObjectSummary objectSummary : objectListing.getObjectSummaries()) { System.out.println(objectSummary.getKey() + " (size: " + objectSummary.getSize() + ")"); }// w w w . jav a2s. c o m }