Go and S3

How to store and consume S3 objects from Golang.
Share this page:

We will be using an S3 bucket to store the Photos. These photos are submitted when user submits a new project.

We already created www.unicornpursuit.com S3 bucket using AWS CDK. You can find more info here.

We will now use AWS SDK for Go, to upload the image to the s3 bucket, and retrieve the URL, which we will then store to a Database. The desired functionality is for a user to click on a project, retrieve all project info from a DynamoDB, and present all the project information,including the image.

If you’ve just landed here, we’re doing a “Become a Cloud Architect” Unicorn Workshop by building a Unicorn Pursuit Web App step by step, and you’re more then welcome to join!

Code

Before we start, let’s understand what we’re building. Check out the diagram below:

S3

In our template folder, we created a project.html file, and we will route “project view” requests to this it. The route will be defined in routes.go package.

We will be retrieving all the info from a database in payload struct. To show the image, we will use payload.Photo, which is a database “column” we store the URL for our Image stored in an S3 bucket:

<!--Display the photo of the project-->
<img src={{.payload.Photo}} alt="Project Photo" style="width:500px;height:300px;">

In handlers.project.go package, which is where we handle Project related functions, we now need to add a functionality of uploading to the S3 bucket, within the createProject function.

We will use the S3 Uploader, which is much more performant then the old Put function. We need to import only s3sessions package. Notice that we’re not using the s3 package:

"github.com/aws/aws-sdk-go/service/s3/s3manager"

First, we need to form and open the uploaded file, from HTML create-project.html we’re getting the type file:

<div class="form-group">
  <label for="file">Select a Photo</label>
  <input type="file" name="file" id="file" accept="image/*"/>
</div>
  fileHeader, err := c.FormFile("file")
    if err != nil {
   c.HTML(http.StatusBadRequest, "create-project.html", gin.H{
    "ErrorTitle":   "File Formation error",
    "ErrorMessage": err.Error()})
     return
  } else {
   fmt.Println(fileHeader.Filename)
  }

  f, err := fileHeader.Open()
  if err != nil {
   c.HTML(http.StatusBadRequest, "create-project.html", gin.H{
    "ErrorTitle":   "File Opening error",
    "ErrorMessage": err.Error()})
     return
  }

Cool, now that we’ve got the file stored in the f variable, let’s now Upload our Image to a bucket:

  // Create an S3 Uploader
  uploader := s3manager.NewUploader(sess)

  // Upload 
  result, err := uploader.Upload(&s3manager.UploadInput{
   Bucket: aws.String(bucket),
   Key: aws.String(fileHeader.Filename),
   Body: f,
  })
  if err != nil {
   c.HTML(http.StatusBadRequest, "create-project.html", gin.H{
    "ErrorTitle":   "S3 Upload Failed",
    "ErrorMessage": err.Error()})
  } else {
   // Success, print URL to Console.
   ///"result.Location is the URL we will be storing at Image field when creating our DynamoDB Project Item"
   fmt.Println("Successfully uploaded to", result.Location)
  }

That’s it, our photo is uploaded. We’ll take care of the permissions once we containerize the App in Fargate, for now - we can keep the Photos with public access.

Where to find more info

Relevant links:




Last modified June 8, 2020: AWS Diagrams Added (c98aff0)