Dev Environment

How to build your Dev environment: Tools, GitHub, AWS CLI, AWS CDK

If you’ve just landed here, we’re doing a “Become a Cloud Architect” Unicorn Workshop by building a Unicorn Pursuit Web App step by step, and you’re more then welcome to join!

MacOS Dev Environment

Before you start doing anything, you should set up your work environment. I’m going to assume you’re working on a MacOS, but if you’re not - Linux and Windows most of the tools have an equivalent. You may also check a blog post about building a decent environment on Widnows using WSL:

GitHub

Source Control Management is essential for the DevOps type organizations.

  • MS has GitHub.
  • GCP has Source Repositories
  • AWS has Code Commit.

SCM Functions are:

  • Fork: Copy the repository to local file system
  • Clone: Pull a copy of a Source Code, which you need to STAGE in order to COMMIT.
  • Commit: Change the LOCAL code
  • Push: Upload committed changes to GitHub
  • Pull requests: like push with needed review before a merge of a branch.
  • Branches: Separation of code change streams, you can have sub-branches, each changing, before you merge it back with the main branch
  • Stage: when you change the code, you can stage individual changes before you push them.

Typically before making any changes, you should pull the current repo:

git pull

To create new branch called “dev” and switch to it:

git checkout -b dev

If it already exists, just switch to it using:

git checkout dev

Push stuff to new branch on Git

git push origin dev

See all the branches

git branch -a

Safest way to merge from Dev to Master

Let’s assume your dev branch is called “dev”:

git checkout master
git pull origin master
git merge dev
git push origin master

Can anybody push to my project on github

No, but if the repository is public others can fork it, commit to their own fork. They can then ask you to pull some of the changes in their fork into your repository via a pull-request. Nobody can push directly to your repository if you are not already granting them write access.

Golang

brew install go

Don’t forget to set path correctly! I’ll assume you installed using “homebrew”, which made a /.go/ hidden folder in your $HOME. Since Catalina is using zsh by default, you’ll want to add these to the path permanently, so do:

nano ~/.zshrc

Add the following, and Save the file:

# Adding GOPATH and GOROOT
export GOPATH=$HOME/.go
export GOROOT=/usr/local/opt/go/libexec
export PATH=$PATH:$GOPATH/bin
export PATH=$PATH:$GOROOT/bin

And restart the terminal, or source .zshrc. To be sure everything is all right, you can open a new Tab in your iTerm and do env to be sure GOPATH and GOROOT point to the right place.

AWS CLI

AWS CLI is a must, because you will be using it a lot. You can find the instructions here, and it should more or less look like following.

Once you’ve installed AWS CLI, test it:

➜  matscloud git:(dev) ✗ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".
➜  matscloud git:(dev) ✗

Nope… so, let’s do what they say. Let’s assume you’re not using AWS SSO which is awesome, and we need to authenticate using IAM Access Key. You need to:

  • Go to AWS Console
  • Choose IAM
  • Click on “Users”, and choose your user.
  • Go to “Security Credentials”, and “Create Access Key”

IMPORTANT: Please make sure you’re not using your access keys ANYWHERE in the code. In fact, each time you go to IAM and create an Access Key, just use it that one time, and delete afterwards. Next time you need it, you can create it. We will go deeper into how to make this less manual labor when we get to IAM part of the workshop.

Let’s configure our AWS CLI. I’m configuring Ireland as my Region by default, and YAML as my output format:


➜  matscloud git:(dev) ✗ aws configure
AWS Access Key ID [None]: YOUR_ACCESS_KEY
AWS Secret Access Key [None]: YOUR_SECRET_KEY
Default region name [None]: eu-west-1
Default output format [None]: yaml
➜  matscloud git:(dev) ✗
➜  matscloud git:(dev) ✗
➜  matscloud git:(dev) ✗ aws s3 ls
2019-11-29 14:13:04 appstream-app-settings-eu-west-1-057097267726-paih1jmm
2019-11-29 14:13:04 appstream2-36fb080bb8-eu-west-1-057097267726
2020-04-14 15:14:34 cdktoolkit-stagingbucket-1ayqaggmn7gsj
2020-04-14 15:30:09 cdktoolkit-stagingbucket-v2756l61ez1l
2018-12-06 21:49:20 cf-templates-1u9btd20s3yn-eu-west-1
2018-11-27 13:33:48 cf-templates-1u9btd20s3yn-us-east-2
2018-12-13 10:50:26 matscloud-images
2020-04-17 23:09:23 s3-prod-unicorn
etc.

Now, let’s try something insane, to get all AWS Services we’re actually using at the moment (watch the time period!), and check out the results:

➜  matscloud git:(dev) ✗ aws ce get-cost-and-usage --time-period Start=2020-04-01,End=2020-04-28 --granularity MONTHLY --metrics "UsageQuantity" --group-by Type=DIMENSION,Key=SERVICE Type=DIMENSION,Key=REGION --output yaml

You might not get much sense out of the results at first, but the use case here is to show you the power of CLI.

AWS CDK

AWS CDK brings the best of both Declarative programming (describe the desired final state) and Indicative programming (code step by step what to do).

We will be using Python for both, CDK and SDK. In my experience, even being much less documented then TypeScript, it works so much better.

To get started you need to set up AWS CDK on your PC using the official link, or just follow the commands below. Just make sure if your “python” command refers to your python 2.X which is dissapearing, or Python 3, which we will be using here. You might need to use “python3” and “pip3” commands.

First make sure which pip you’re using, pip or pip3:

➜  matscloud git:(dev) ✗ pip --version
zsh: command not found: pip
➜  matscloud git:(dev) ✗
➜  matscloud git:(dev) ✗ pip3 -V
pip 19.0.3 from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/site-packages/pip (python 3.7)

Ok, so I’m on pip3. Now:

python3 -m ensurepip --upgrade
sudo pip3 install --upgrade virtualenv

Create a new project. This will be your test project, for everything you want to try during the Unicorn workshop.

Go create a IAC (Infrastructure as Code) Folder, get in that folder, and start… install NPM and CDK.

➜  iac git:(dev) brew install npm
...
➜  iac git:(dev) npm install -g aws-cdk

/usr/local/bin/cdk -> /usr/local/lib/node_modules/aws-cdk/bin/cdk
+ aws-cdk@1.36.0
added 213 packages from 187 contributors in 28.754s
➜  iac git:(dev)
➜  iac git:(dev)
➜  iac git:(dev) cdk --version
1.36.0 (build 47c9919)
➜  iac git:(dev)
➜  iac git:(dev) pip install --upgrade aws-cdk.core
...
➜  iac git:(dev)

You’re good for now!


Last modified May 19, 2020: NextLessionAdded (2213181)