Python: Parsing and Manipulating JSON Data

Guides, Python /

Understanding how to parse and manipulate data is a foundational skill to have in network automation and programmability. In this blog post, we will be taking a look at basic parsing and manipulating of JSON (JavaScript Object Notation) data, with a networking context. I have also given a real-world example of using JSON data and Python objects utilizing the always-on DevNet IOS-XE sandbox.

There is a GitHub repository to go along with this blog post located on my GitHub profile. I encourage you to clone the repository and get your hands dirty while following this guide. The code for this post is located in the JSON directory. I will have future posts going over how to work with YAML and XML data, which is important for NETCONF use cases.

Clone the Repo!

The Sample Data

The sample JSON data we’re working with is some interface data containing interface names, descriptions, whether it’s enabled or not, and IPv4 information.

All JSON data begins with { and ends with a }. Notice how all of the interface data is within those two characters. JSON contained key:value pairs. There is only one object in this JSON sample, which is interfaces. Interfaces’ value is a list, denoted by the [ and ] characters, of objects, one for each interface.

Within each interface object, there are key:value pairs with data about the interface. Notice how there is more data nested within the “ipv4” key. The ipv4 key’s value is another object, which consists of two key:value pairs for the IP address and subnet mask.

{
    "interfaces": [
        {
            "name": "GigabitEthernet0/0",
            "description": "Uplink 1",
            "enabled": true,
            "ipv4": {
                "address": "10.10.1.1",
                "mask": "255.255.255.0"
            }
        },
        {
            "name": "GigabitEthernet0/1",
            "description": "Uplink 2",
            "enabled": true,
            "ipv4": {
                "address": "10.10.2.1",
                "mask": "255.255.255.0"
            }
        },
        {
            "name": "Loopback1",
            "description": "Loopback",
            "enabled": true,
            "ipv4": {
                "address": "1.1.1.1",
                "mask": "255.255.255.255"
            }
        }
    ]
}

Import json

In order to work with JSON in Python, we need to use built-in JSON package. Python supports JSON natively. All you need to use it is add the import json statement at the top of the your python module.

The JSON package as several functions to be aware of. These are used to load and write JSON data:

  • load() – Import native JSON from a file and convert to it to a python object.
  • loads() – Load string. Import JSON data from a string for parsing and manipulating within a script. T his cannot be used with file objects.
  • dump() – Write JSON formatted data from a python object into a file.
  • dumps() – Dump string. Take JSON dictionary data and convert to a serialized string for parsing and manipulating.

Let’s take a look at how we’re using these functions in the my sample code. We can use json.load() to parse the JSON data directly into a Python object, which is being setting to “interface_data”. This is a great way to load JSON formatted data from a file. I am using a context manager for file handling, which is not in the scope of this post. Check out this GeeksForGeeks post for more information on handling files in Python.

with open("interfaces.json") as data:
    interface_data = json.load(data)

We can also read the data in using the read() function. This will create a JSON formatted string, which I am setting to “interface_data_string”. Because this is a string and not a python object, we’ll need to use json.loads() to convert it from a string to a Python object.

with open("interfaces.json") as data:
    interface_data_string = data.read()

interfaces_string_object = json.loads(interface_data_string)

Manipulating the Data

Now that we’ve got the JSON data loaded into a couple of variables, how can we work with this data? The first thing we need to do is make sure we know how to access elements in the dictionary object.

We will want to start out with the python object that holds the data, which is interface_data. We can then add reference key names inside brackets, for example [“interfaces”], which will give access to the items inside the interfaces list. Because interfaces is a list, we need to use list indices, or positions, to navigate it. 0 is the first object in the list, which is the object that holds GigabitEthern0/0 information. 1 is the position of the second object, which holds information for GigabitEthernet0/1. The object that holds information about the loopback interface is at list position 3. If we had more interfaces, they would continue with 4, 5, 6, etc.

After specifying the list index, we can tack on the key of whatever we’re wanting to print or edit, in this case I am printing the descriptions of both GigabitEthernet interfaces. To edit the descriptions descriptions, all we have to do is set it to what we want it to be. In the example below, I am changing the descriptions on both interfaces from “Uplink 1” and “Uplink 2” to “Primary Uplink” and “Secondary Uplink”. We can print that again to verify the change was made.

print(interface_data["interfaces"][0]["description"])
print(interface_data["interfaces"][1]["description"])

interface_data["interfaces"][0]["description"] = "Primary Uplink"
interface_data["interfaces"][1]["description"] = "Secondary Uplink"

print(interface_data["interfaces"][0]["description"])
print(interface_data["interfaces"][1]["description"])

Accessing IPv4 data is also very simple. Just like we did for description, the only thing we need to do is tack on “ipv4” after the list index. This will get us down to the object that holds the IP address and subnet mask. From there, all we need to do is add either “address” or “mask” to print or edit those values. In the example below, I am changing the IP address and mask of GigabitEthernet1 and then printing to very the change was made.

print(interface_data["interfaces"][0]["ipv4"]["address"])
print(interface_data["interfaces"][0]["ipv4"]["mask"])

interface_data["interfaces"][0]["ipv4"]["address"] = "10.10.1.50"
interface_data["interfaces"][0]["ipv4"]["mask"] = "255.255.255.252"

print(interface_data["interfaces"][0]["ipv4"]["address"])
print(interface_data["interfaces"][0]["ipv4"]["mask"])

Try adding your own lines of code to print or edit other pieces of information. If you have trouble, feel fre to comment.

Writing the Modified Data

Now that we have some modified data that no longer matches the original data we imported, let’s write this data to a new file. I am continuing to use a context manager for file handling, but I am going to add the “w” option for writing to a file.

To write the JSON data back to a file, I am using the before mentioned json.dump() function, which will take the python object and write JSON formatted data into a file called “interfaces_dump.json”. I am also using the optional indent parameter to help with readability.

with open("interfaces_dump.json", "w") as fh:
    json.dump(interface_data, fh, indent=4)

Real-World Demo

Let’s take a look at a real-world example of using JSON data in python. In this example, I will be using the Cisco DevNet IOS-XE always-on sandbox to retrieve a list of interfaces, and create a new loopback interface. This script uses RESTCONF, which is not in the scope of this guide. However, the important part is that you understand the data we are retrieving, and the payload we are sending to the device to create the loopback interface.

The code for this script is located in the iosxe_sandbox_example.py file inside the GitHub repository.

DevNet Sandbox Info.

This script is using the DevNet IOS-XE always-on sandbox. This sandbox can go offline or have it’s credentials changed at any time. I will do my best to keep the code up to date with a working example.

HOST = "sandbox-iosxe-latest-1.cisco.com"
PORT = 443 (RESTCONF)
USER = "developer"
PASS = "C1sco12345"

Retrieving Interface Data

We can use RESTCONF to retrieve interface data from the IOS-XE device. RESTCONF enables this data to be in JSON format. The code below uses the Python requests library to send a request to the device for interface data, indicated by Cisco-IOS-XE-interfaces-oper:interfaces in the URL.

If the response is good, I use the json() function to parse the response text into a Python object, and then use json.dump() to write that data into a file called “iosxe_interfaces.json”. This file is included in the GitHub repository.

url = f"https://{HOST}:{PORT}/restconf/data/Cisco-IOS-XE-interfaces-oper:interfaces/"
headers = {
'Content-Type': 'application/yang-data+json', 
'Accept': 'application/yang-data+json',}
response = requests.get(url, headers=headers, verify=False, auth=(USER, PASS))

if response.ok:
    response_json = response.json()


    with open("iosxe_interfaces.json", "w") as fh:
        json.dump(response_json, fh, indent=4)

else:
    print(f"Error occurred with status code {response.status_code}")

Creating a New Loopback Interface

To create a new loopback interface on the device, I am making a post request that contains data for the device to use to create the new interface. Pay attention to the payload variable in the below code. Notice that the data inside it is JSON formatted. I am using json.dumps() here because the device expects a string in JSON format.

post_url = f"https://{HOST}:{PORT}/restconf/data/ietf-interfaces:interfaces"
post_headers = {
'Accept': 'application/yang-data+json',
'Content-Type': 'application/yang-data+json',
'Authorization': 'Basic ZGV2ZWxvcGVyOkMxc2NvMTIzNDU='
}

payload = json.dumps(
    {
        "ietf-interfaces:interface": {
            "name": "Loopback510",
            "description": "Added using RESTCONF and Python - nickm155.sg-host.com",
            "type": "iana-if-type:softwareLoopback",
            "enabled": True,
            "ietf-ip:ipv4": {
                "address": [
                    {
                        "ip": "50.10.10.10",
                        "netmask": "255.255.255.255"
                    }
                ]
            }
        }
    }
)
post_response = requests.post(post_url, headers=post_headers, data=payload)
print(post_response.text)

Give running this script a try. If it works, you should receive no error message and have an updated file containing the interface data. You can give it another run to update the file again with the loopback you create. You should get an error message this time saying the loopback already exists. Try changing the loopback values to create new loopbacks as well.

Thanks for Reading

Thanks for checking out this post. If you found it useful, feel free to leave a comment to let me know. If you have trouble or if something is not working as you expect, you can leave a comment or send a message to me through my Contact form.

Leave a Comment

Your email address will not be published. Required fields are marked *