Skip to main content

One post tagged with "Go"

View All Tags

· 7 min read
Iain Sutherland

Previously, I wrote a tutorial about how I, as a proficient C# developer, would go about converting code using the AWS .NET SDK for S3 to instead use the Ionburst .NET SDK. The conclusion of that being that it was a very trivial exercise.

I’ve been asked, could you do the same with the Go language? Well I’m hardly a proficient Go developer, having spent just a few months using it, but I don’t think it will be a massive undertaking. Let’s give it a go.

Prerequisites​

To get started, you'll need appropriate credentials. For this tutorial, I have both AWS SDK credentials and Ionburst Cloud credentials set up on my PC. If you are already familiar with AWS credentials then you may already have a .aws/credentials credentials file in your home directory. For this example, we'll be using credentials setup in the default profile.

The Ionburst Cloud SDKs can also use a credentials file in your home directory, .ionburst/credentials and, like AWS, can also use environment variables. The Ionburst Cloud .NET SDK documentation gives further detail on how the SDK can be configured.

The starting position​

It wasn’t quite as trivial to cobble together a Go program using the AWS Go SDK as it had been for me to write the C# starting position, but I came up with a similar program.

It functions the same. Namely, upload a file called image.jpg to an S3 bucket, fetch the object from the bucket, storing it locally as image_download.jpg, before removing the object from the bucket.

Our simple program:

package main
import (
"fmt"
"io"
"os"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
const (
bucketName = "ibc-example"
)
type storageInterface interface {
UploadFile(fileName string) (bool, error)
FetchFile(fileName string) (bool, error)
DeleteFile(fileName string) (bool, error)
}
type awsClientWrapper struct {
client *s3.S3
}
func main() {
storageClient, err := CreateS3Client()
if err != nil {
fmt.Println("Failed to create storage client")
os.Exit(1)
}
upload, err := storageClient.UploadFile("image.jpg")
if err != nil {
fmt.Println("Failed to upload file")
}
if upload {
fmt.Println("File uploaded")
fetch, err := storageClient.FetchFile("image.jpg")
if err != nil {
fmt.Println("Failed to fetch file")
}
if fetch {
fmt.Println("File fetched")
delete, err := storageClient.DeleteFile("image.jpg")
if err != nil {
fmt.Println("Failed to delete file")
}
if delete {
fmt.Println("File delete")
}
}
}
}
func CreateS3Client() (storageInterface, error) {
sess, err := session.NewSession(&aws.Config{
Region: aws.String("eu-west-1")},
)
if err != nil {
return nil, err
}
return awsClientWrapper{client: s3.New(sess)}, nil
}
func (c awsClientWrapper) DeleteFile(key string) (bool, error) {
var deleteInput *s3.DeleteObjectInput = &s3.DeleteObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(key)}
_, err := c.client.DeleteObject(deleteInput)
if err != nil {
return false, err
}
var headInput *s3.HeadObjectInput = &s3.HeadObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(key)}
headErr := c.client.WaitUntilObjectNotExists(headInput)
if headErr != nil {
return false, headErr
}
return true, nil
}
func (c awsClientWrapper) UploadFile(fileName string) (bool, error) {
file, fileErr := os.Open(fileName)
if fileErr != nil {
return false, fileErr
}
var putInput *s3.PutObjectInput = &s3.PutObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(fileName),
Body: file}
_, err := c.client.PutObject(putInput)
if err != nil {
return false, err
}
defer file.Close()
return true, nil
}
func (c awsClientWrapper) FetchFile(key string) (bool, error) {
var fetchInput *s3.GetObjectInput = &s3.GetObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(key)}
fetchOutput, err := c.client.GetObject(fetchInput)
if err != nil {
return false, err
}
parts := strings.Split(key, ".")
var outputFileName string = "download"
if len(parts) == 1 {
outputFileName = fmt.Sprintf("%s_download", parts[0])
}
if len(parts) == 2 {
outputFileName = fmt.Sprintf("%s_download.%s", parts[0], parts[1])
}
if len(parts) > 2 {
outputFileName = fmt.Sprintf("%s.download.%s", parts[0], parts[len(parts)-1])
}
file, fileErr := os.Create(outputFileName)
if fileErr != nil {
return false, err
}
defer file.Close()
_, err = io.Copy(file, fetchOutput.Body)
return true, nil
}

The conversion​

So how do I change that code to use the Ionburst Cloud Go SDK?

Well package management is slightly different in that there’s nothing like NuGet for Go and the “import{}” directive seems to be automatically handled by Visual Studio Code, unlike the C# using statements which are manually maintained.

Anyway, first off I’ll change the structure containing the client. In hindsight I should probably have just called the structure clientWrapper and then I wouldn’t have even needed to change the name. It’s just a case of replacing *s3.S3 with *ionburst.Client:

type awsClientWrapper struct {
client *s3.S3
}
to
type ionburstClientWrapper struct {
client *ionburst.Client
}

Next, I then replace the CreateS3Client function with a CreateIonburstClient function which looks like:

func CreateIonburstClient() (storageInterface, error) {
client, err := ionburst.NewClient()
if err != nil {
return nil, err
}
return ionburstClientWrapper{client: client}, nil
}

With this complete, the first line in main() becomes:

storageClient, err := CreateIonburstClient()

Next, the functions have to be changed to use ionburstClientWrapper instead of awsCientWrapper, and the final step is to change the places where a function call is made to c.client because that has changed from *s3.S3 to *ionburst.Client.

Using Visual Studio Code, I was already being presented with the squiggly red lines to show me the functions that were not now defined for *ionburst.Client.

With the Ionburst Cloud Go SDK there are no request and response structures, everything is done with parameters and discrete return values. The UploadFile function ends up a bit smaller, becoming:

func (c ionburstClientWrapper) UploadFile(fileName string) (bool, error) {
file, fileErr := os.Open(fileName)
if fileErr != nil {
return false, fileErr
}
err := c.client.Put(fileName, file, "")
if err != nil {
return false, err
}
defer file.Close()
return true, nil
}

It’s worth noting that the Go SDK Put function does require a classification to be specified. In the event a classification isn't supplied, a default is applied, so I've left it empty.

The FetchFile and DeleteFile required similar changes; c.client.GetObject() becomes just c.client.Get(), and in the delete function the client call becomes c.client.Delete(). For both, the request structure can be removed and the return values are not wrapped in a structure.

Conclusion​

So I would say that for a proficient Go developer, migrating from the AWS Go SDK for S3 to the Ionburst Cloud Go SDK would be as trivial an exercise as I experienced doing the same thing with .NET.

In terms of coding time, it did take less time to make the changes to use the Ionburst Cloud Go SDK than it did to develop the sample program, but since I have less experience with Go, the original program was the more difficult undertaking, while the conversion was just changing a few lines of existing code.

The full converted code can be found below:

package main
import (
"fmt"
"io"
"os"
"strings"
"gitlab.com/ionburst/ionburst-sdk-go"
)
const (
bucketName = "ibc-example"
)
type storageInterface interface {
UploadFile(fileName string) (bool, error)
FetchFile(fileName string) (bool, error)
DeleteFile(fileName string) (bool, error)
}
type ionburstClientWrapper struct {
client *ionburst.Client
}
func main() {
storageClient, err := CreateIonburstClient()
if err != nil {
fmt.Println("Failed to create storage client: ", err)
os.Exit(1)
}
upload, err := storageClient.UploadFile("image.jpg")
if err != nil {
fmt.Println("Failed to upload file")
}
if upload {
fmt.Println("File uploaded")
fetch, err := storageClient.FetchFile("image.jpg")
if err != nil {
fmt.Println("Failed to fetch file")
}
if fetch {
fmt.Println("File fetched")
delete, err := storageClient.DeleteFile("image.jpg")
if err != nil {
fmt.Println("Failed to delete file")
}
if delete {
fmt.Println("File deleted")
}
}
}
}
func CreateIonburstClient() (storageInterface, error) {
client, err := ionburst.NewClient()
if err != nil {
return nil, err
}
return ionburstClientWrapper{client: client}, nil
}
func (c ionburstClientWrapper) DeleteFile(key string) (bool, error) {
err := c.client.Delete(key)
if err != nil {
return false, err
}
return true, nil
}
func (c ionburstClientWrapper) UploadFile(fileName string) (bool, error) {
file, fileErr := os.Open(fileName)
if fileErr != nil {
fmt.Println("File open error: ", fileErr)
return false, fileErr
}
err := c.client.Put(fileName, file, "")
if err != nil {
fmt.Println("Ionburst upload error: ", err)
return false, err
}
defer file.Close()
return true, nil
}
func (c ionburstClientWrapper) FetchFile(key string) (bool, error) {
fetchReader, err := c.client.Get(key)
if err != nil {
return false, err
}
parts := strings.Split(key, ".")
var outputFileName string = "download"
if len(parts) == 1 {
outputFileName = fmt.Sprintf("%s_download", parts[0])
}
if len(parts) == 2 {
outputFileName = fmt.Sprintf("%s_download.%s", parts[0], parts[1])
}
if len(parts) > 2 {
outputFileName = fmt.Sprintf("%s.download.%s", parts[0], parts[len(parts)-1])
}
file, fileErr := os.Create(outputFileName)
if fileErr != nil {
return false, err
}
defer file.Close()
_, err = io.Copy(file, fetchReader)
return true, nil
}