Python - Libraries for DevOps Engineers

▶Python Libraries

Python has a huge of libraries and modules that are commonly used by DevOps engineers in their daily tasks.

Here are some of them:

Pandas
NumPy
Keras
TensorFlow
Scikit Learn
Eli5
SciPy
sys
JSON
Yaml
Logging
Argparse
Requests

- JSON

JSON stands for JavaScript Object Notation.

It is a lightweight data-interchange format that is easy for humans to read and write and easy for machines to parse and generate.

JSON is a text format that is completely language-independent, meaning that it can be used to represent data structures in any programming language.

JSON data is represented as key-value pairs, with the keys being strings and the values being of various types, including strings, numbers, arrays, and objects.

Example:

{
    "name": "Fran",
    "age": 44,
    "email": "fran@example.com",
    "phone_numbers": [
        {
            "type": "work",
            "number": "xxx-xxxx"
        }
    ]
}

In this example, we have an object with some key-value pairs.

- YAML

YAML (short for "YAML Ain't Markup Language") is a human-readable data serialization format.

It is often used for configuration files, but can also be used for data exchange between languages and platforms.

YAML is designed to be easy to read and write, with a syntax that is more concise and readable than other markup languages like XML or JSON.

YAML is whitespace-sensitive, meaning that indentation is used to indicate structure and hierarchy within the data.

Example:

name: Fran
age: 44
address:
  street: Av Liberdade, x
  city: Lisbon
  country: Portugal
  zip code: xxxxxxx

The file consists of a series of key-value pairs, where the keys are strings and the values can be any valid data type, including strings, numbers, booleans, arrays, and objects.

▶Task 1

Create a Dictionary in Python and write it to a JSON File.

I have created a file name my_dict.py, and I have written a file into it and will import it into JSON File:

import json

my_dict = {
       "name" : "Fran",
       "age" : 44,
       "city": "Lisbon"
}

with open( "my_dict.json", "w") as file:
       json.dump(my_dict, file)

Create a JSON file by running this Python file and verify with the command ls *json

Check the file using vi my_dict.json:

▶Task 2

Read a JSON file services.json kept in this folder and print the service names of every cloud service provider.

json.load(): this function accepts the file object, parses the JSON data, populates a Python dictionary with the data and returns it back to you.

import json
with open("services.json") as f:

    data = json.load(f)
    print("aws:",data[ 'services']['aws']['name'])
    print("azure: ",data['services']['azure']['name'])
    print("gcp: ",data['services']['gcp']['name'])

Read the YAML file using Python, file services.yaml and read the contents to convert YAML to JSON.

We can read the YAML file using the PyYAML module’s yaml.load() function. This function parse and converts a YAML object to a Python dictionary (dict object).

To install the PyYAML module, you’ll need to run the pip command, which is the package installer for Python. The below command installs the PyYAML module.

pip install pyyaml

Below code read services.yaml file using Python.

First import yaml module using an import statement
Read the file using the open method
safeloader method read the file content and converts it to a dictionary Python object

Convert YAML to JSON:

json.dumps() function will convert a subset of Python objects into a JSON string.

import json
import yaml

# Open the files and load the files

with open('services.yml', 'r') as f:
         data = yaml.safe_load(f)
         print(data)

with open('services.json', 'w') as json_f:
         json.dump(data,json_f)

final_output = json.dumps(json.load(open( 'services.json')), indent=2)
print("json_file :\n", final_output)

This was a nutshell of Python and its Libraries.

I hope this article is helpful to someone.

Python - Libraries for DevOps Engineers

#90DaysofDevOps Challenge - Day 15

Table of contents