Simple method for removing an S3 bucket with files present Clark Wilkins, Simplexable 2022.01.20

In the design process for topical 2.0, I ran into a problem. Starting in this version, when a user attaches a file to the discussion, a bucket is created on S3 with the discussion ID. The file is loaded in using the UUID generated for the new discussion "comment". The advantage of this is we can store a reference that has the discussion ID (same as the bucket) and the comment ID (same as the item name in the bucket).

This makes it very easy to set up a call to S3 in the Node/Express API to remove the single item. However, deleting the topic is a little more involved. We need to remove all the contents of the bucket, and only then remove the bucket itself. This is further complicated by AWS execution timing.

This sequence does not work:

  1. get a list of the files as an array
  2. await a deleteObject to run
  3. await a deleteBucket to run

What happens every time is AWS throws an error on step 3, saying the bucket is not empty, but deleting the bucket anyway. The solution is to use promise chaining.

let { discussion, owner } = req.body;
 


var params = {

  Bucket: S3_BUCKET,

  Prefix: discussion + '/'

}



const s3 = new aws.S3();

const { Contents: files } = await s3.listObjects( params ).promise();



if ( files.length > 0 ) {



  const Objects = Object.entries( files ).map ( theFile => {



    const { Key: key } = theFile[1];

    return { Key: key }



  });



  params = {

    Bucket: S3_BUCKET,

    Delete: {

      Objects: Objects

    }

  }



  await s3.deleteObjects( params )

    .promise()

    .then( async () => {



      params = {

        Bucket: S3_BUCKET

      }


 
      await s3.deleteBucket( params ).promise();



    } );



}

Breaking this down:

  1. Get the contents of the bucket (which has the same name as the discussion ID) as files.
  2. Use Object.entries( files ).map( ... ) to gather all of the filenames as keys into a new Objects object. Here, we are structuring this to conform to the AWS parameter expectations for deleteObjects.
  3. Delete all files in Objects using deleteObjects.
  4. Invoking promise chaining via then, we wait for deleteObjects to finish, and only then call for deleteBucket via an asynchronous bubble (anonymous) function.

This avoids the timing problem by making sure the bucket delete does not happen before deleteObjects finishes. This is not very well explained in most of the docs I read, so I have offered a more "legible" version above.