Posts in DevOps

3 Rules For Cloud Security

What is your cloud security approach?

When designing a product to work on the cloud it’s best practice to include IT and Cloud security in the product runtime, infrastructure and operations.

The Challenge

When using cloud the approach needs to be different than on-perm or just consuming SaaS from another provider, it’s very easy to open ports and permit access to cloud resources, and because it’s in the “cloud” it might be accessible from public and external networks.

Keeping track of modifications or preventing admins and developers access to modify resources can hinder the normal operation of IT and Development, so it’s better to implement a different approach.

An approach that is a mindset of Cloud security considerations in every project and modification, changes are necessary in order to improve and develop the product you’re working on.

Authentication

Authentication means: who are you?

Examples of identify in roles and positions:

  • admin
  • developer
  • contractor
  • customers
  • etc..

Authorization

Authorization means: What can you do?

Examples of permissions:

  • add users
  • delete users
  • add new clients
  • open security-group ports
  • download files
  • access resources (databases, servers)
  • etc..

Connection

Connection means: Where are you connecting from?

Example of connections:

  • Official HQ Offices
  • Remote workers (VPN)
  • Customers (anywhere)
  • Private-Link
  • etc..

How to successfully have a secure cloud account?

Choose the best suited approach for you and your team and implement that approach as a mindset, the approach with those 3 recommended rules is easy to remember and easy to implement.

Do you maintain a regular cloud security operation?
Do you know the status of your cloud security?
If your answer is yes, than contact us now and we’ll do the cloud security for you.
To contact us click HERE

How software update freeze can make your stack obsolete

Do you update your software frequently?

Is software update part of your CI/CD pipelines?

What is continuous update?

The issue with hard-coding software versions and not updating

Just to clarify this post is relevant to 3rd-party software and packages you import to your application (via apt, yum, pip, gem etc.. or downloading binary .jar etc..) also for OS versions

The wrong approach in my opinion is to statically add version numbers to imported packages and use the same OS version throughout your infrastructure and code

Why you ask?

Once your code work with other software (3rd-party) and tests are ok, you assume that the process is complete and resume working on your code

everything works until it doesn’t !

Scenario 1

let’s say you’re using java and a vulnerability is discovered and fixed with a new release, now you need to upgrade to new release but your runtime version is too far behind the latest version and cannot be upgraded

or better yet, you can upgrade but other components that communicate with your code is not compatible with the latest version

what do you do? oh yes, revert !

Scenario 2

the operating system is a few versions behind the latest, let’s say Ubuntu 18.04 and now you want to use Ubuntu 22.04 and your code is python 3.6

guess what Ubuntu 22.04 does not ship with the same python as Ubuntu 18.04

now you need to compile python 3.6 from source and install it to Ubuntu 22.04 and make sure to update the PATH to use python 3.6

Backlog

so now you decided to use python 10 instead of python 3.6 but what about pip packages? they are probably not compatible, why? because pip use the python version too

now go over your entire code and make sure every function works with new python version, then you’ll probably decide it’s too much work right now and not to upgrade

Solution

Simple, don’t freeze software updates!

if you keep your software up to date (including your OS) it forces you to adapt as you go! no need to upgrade or schedule upgrades because it’s a mindset, your software is evolving

stopping your software from evolving does not make sense, in fact it’s the opposite from what your job description is… developer

How to keep your software with latest version?

The answer is CD (continuous delivery)

CD means how fast, reliable and how frequent you deploy your code to production

So the goal is to deploy to production whenever you want and a few times per day, if you do that you know your code is in a releasable state

So using the latest release software while keeping a releasable state will make your job easier and your product better

Deploy vs. Release

What is the difference between deploy and release?

when you’re working on servers and operating a runtime of application you need to make sure the deliver of new software to the users

doing so there are two option

  1. just deploy the new software to the servers
  2. deploy the new software but don’t make it available yet

Deploy

Deploy means you put the new software on the servers

Release

Release means that new software is now available to users

Two step process

making the deploy and release a two step process will ensure a smooth integration of new software into production servers, yes you can still can deploy after testing and make the two step process a one step process

How can you deploy and not make the new software operate instead of the current software?

using feature flags

with feature flags you can deploy the new software but not make it available yet, after you made a decision on when to activate the new software you can make the feature flag active and expose the new software to users (release)

a feature flag is just an if else statement on the new function of the new software

10 tools for cloud admins

As a cloud admin your work is broad and you need tools to improve your work and efficiency.

Here’s the best 10 tools in my opinion every cloud admin should have in their tool belt, or their laptop in our case.

This is my list and the best tools in my opinion.

Code

1. VS Code is one of the popular code editors and it is my preferred code editor, it has many plugins and you can customize it to your preferences

2. Python is the easiest and most robust coding tool for cloud admins, with plugin support to almost any cloud.
no need for introduction here

3. boto3 is specific for AWS but still worth mentioning if you’re using AWS as your cloud.
it have a very good documentation and does the job, it’s very important when you integrate automation with your projects.

Servers

4. Docker is by far the fastest way to install something, it’s just docker run image-name and the app is installed, although it’s not recommended for production it is suitable for IT servers, just remember to backup the volumes.

5. OpenVpn is a great tool to encrypt the connection to your servers, it can also save some public IP addresses since you connect to the server’s private IP addresses.

6. Sensu is great to know what’s up with your servers, get the metric you want to know and get alerts for something you need to know

7. Puppet is your preferred option if you don’t use Docker in your servers, it can configure your servers and maintain a state for your servers, easy to use and plenty of modules to choose from.

Misc

8. Zsh is better then the default terminal consoles and is can be used in your laptop and in your servers, install it on your servers as well to get the experience you get on your laptop.

9. Let’s Encrypt made it so easy to secure your public connections to your apps, there’s is really no reason why not to use it for your apps.

10. KeePass is a password manager that you need because you connect to many servers and services and it’s a great tool for this task, just remember to backup KeePass database.

Build, Test, Deploy. the hello world of CI/CD

When you start at programming the traditional first program you learn to write is: hello_world() and it became quite common on every programming language you choose to learn.

The hello world of CI/CD

In CI/CD you probably noticed the term build > test > deploy > repeat and this is the hello world in DevOps.

But what does it means?

so let’s assume your code is in Python, then there’s nothing to build unlike Go (golang) so why do you need build stage?

Let’s start from end to start

let’s assume your app is already deployed and running at your cloud of choice, how do you update your app to a newer code?

  • first build your code, why build? because it’s easy and convenient to use and deploy on different environments and can be reused or rollback if needed.
  • second test your code, why test? because you want to ensure your app runs without problems or bugs (making sure code does what it needs to do)
  • third deploy your new code (new version), so the users that use your app can have the new features or the newer bugs (that’s a joke) available for use.

How do you write your first CI/CD hello world?

  1. write code in your prefer language of choice
  2. build it via docker, and have a docker image from it (use tag for versions)
  3. run tests on the functions of the code using the built image (from step 2)
  4. deploy the new image after a successful test via docker with new image tag

so have fun with CI/CD and solve this problem for your company

How many stages do you need for a Jenkinsfile

You need to build a new Jenkinsfile / pipeline to build and test an application or a feature, so how many stages / steps do you use?

Also do you need a few stages or one stage is enough?

Here is where your approach comes to play, there are two main approaches I noticed in workflows from people

1st option: one stage to build and deploy

let’s assume you need to build an artifact from your code so one stage can be enough, something like: build your artifact > deploy (upload to your repository)

this approach will complete a couple of goals

  1. your code is built automatically via scripted job and if the build passes you know part of the code is OK, will it deployed successfully on your server? maybe, so you may want to add install stage before deploying.
  2. once your code built now it’s the easy part to upload the artifact to your repository or registry if it’s a Docker image.
    so it’s easy to understand and because it only two stages it’s faster to complete but you might want more options to integrate in your build.

2nd option: many stages to build, test and deploy

this approach will probably cover most of your software needs

  1. you build your code and once it’s done you start the second stage and start testing, now you don’t need a test to build because that already being done so you can add the installing stage and just deploy your artifact on a docker machine to verify it can be installed as predicted.
  2. once your artifact is installed you can continue to the next stage and add more test in a separate stage, here is where you might want to add commands to check some functionality that your code needs to do.
  3. after the first phase of your tests is completed you can add another stage to deploy your artifact after install to a light environment like Docker or a testing server (Jenkins agent) and make an API call to test some functions to verify that your code is working as expected.
  4. Now it’s where the parts come to play together, add another stage of tests to make sure your code can integrate with 3rd parties like Email, Slack or external API

once your tests are OK it’s deploy time

Time to deploy your code / artifact to your production server, it is recommended that you deploy it first to a staging server, just to make sure you didn’t miss anything, staging will run with your new code just a few minutes before deploying to production.

once everything is OK with your staging deploy to prod, a few options here:

deploy your code instead of the current version, until your deployment will complete your service will probably be unavailable temporary.

add another server to your servers group with the new version and once is OK remove the old version from the old server.

if your app have considerable traffic you might want check deployment tools like Spinnaker to deploy a canary approach of your app.

P.S.

use your approach based on your app needs and traffic

Continuous delivery in a bite

Continuous Delivery AKA CD is a set of practices that aim to build > automate > release software cycles.

CD is about delivering software to the end-user as fast as you can it’s a challenge due to best practices and other software concerns.

Benefits of CD

Let’s go over a few benefits you can achieve in your CD process/workflow.

Faster time to deliver software

Once your development team agrees on a CD “plan” something that works for your product and company and you have “field-tested” it, you’ll have faster build and release cycles.

Better teamwork in the development

Again let’s use the word “agreed” which means your development team has a plan and a system to accomplish a version release cycle, your development team will work better towards that goal of developing and releasing new software, why? fewer arguments, people know the workflow, and stuff gets done.

Lower costs (in theory)

Let’s assume you have the best optimal workflow for your development team, now that your development cycles are working fairly faster the costs may be lower, less compute resources, fewer meetings, and more efficiency.

Faster reach to market

So this is more of a business aspect but is depended on product development, if your product development is fast meaning you can release versions of your product (in this case software) faster then you can deliver faster features and products thus giving you the advantage over competitors that have a slower rate of product release.

Immutable servers do not accept changes

Do you have long-running servers? yes, keep reading.

What is a long-running server?

A long-running server is a compute resource that is hardcoded to perform a certain task, the caveat is that this resource can be changed over time and re-configured the state of the runtime.

Why do you care about changing the runtime state?

A few issues may pop when using mutable servers:

  • who knows the current state of the servers?
  • how do you know if things are still working as intended?
  • how do you know who logs in?
  • how do you keep track of who did things on the servers?
  • how do you keep the consistency of your servers?

Wait! don’t answer those questions because you will probably have a sophisticated answer unless you have a system that is working for you and is proven to manage a large number of servers.

Enter Immutable servers (compute resources)

Now that we have established the issues with long-running servers let’s talk about Immutable servers AKA immutable compute resources. the core concept is that immutable servers don’t accept changes thus making the state of the current runtime precise to what you wanted to be deployed.

by using this method of immutable servers you can expect the same result of operation across your servers or compute resources.

How to automate jobs using Jenkins agent

Do you maintain Jenkins agents for your jobs? does your Jenkins agents running while in idle?

What if you could start a Jenkins agent for every job you run at Jenkins, and the best part no need to maintain and keep a Jenkins agent like a long-running server.

So how do you automate your jobs?

  • Start with installing the EC2 and Cloud plugins in your Jenkins server.
  • then configure Jenkins to run jobs only at agents and not on the master.
  • start a new job and a new instance will start (I’m using AWS so an instance will be initialized on that EC2 cloud)
  • wait for about 5-10 (it varies) until the agent is connected and watch the job run at that specific agent/node.

also you can configure the Cloud plugin to terminate the Jenkins agent once the agent is in idle for “x” amount of time, depends how many jobs you run every hour.

The outcome

If you use this approach you can save on cloud costs and automate jobs

P.S. you can configure spot instances to run as Jenkins agents.

Do you want to automate your Jenkins agents?

I can automate your Jenkins agents for you, contact me today HERE

What is your Deployment Strategy

As a DevOps and a cloud administrator you need to deploy apps to your production cluster.

How do you deploy your apps?

There are many options and there is no perfect one because every app and every company has a different approach so in this blog g post we’ll talk about a few so you can choose the deployment strategy that is suitable for you.

The purpose of the deployment is to deploy a new version of your app and images.

Rolling Red / Black or Green / Blue

In this approach, we will be Rolling the new version per instance or per group of instances.

Canary Analysis

In this approach we will do the following steps:

  • Replace a small number of instances
  • A certain percent of traffic is sent to the new version
  • Meant to test a stable version

Red / Black or Green / Blue

In this approach we will do the following:

  • Starts a duplicate server group
  • When the new group is OK traffic is sent to this new group

Choosing your deployment strategy can help you with deploying every new version of your software.