An Updatable JSON Database

In this post, we will be looking at how to create an updatable JSON database in Python. Why might this be useful? If you have a large quantity of data that you need to manually input, writing perfect JSON can sometimes be tedious. By writing a few simple functions, you can radically increase your productivity by simply inputting the data in the terminal. In this post, we will make functions that allow a user to see the existing JSON data, delete entries, and edit entries. At each stage I will provide a video to explain the steps.

In this post I break the cardinal rule of programming on purpose. I repeat blocks of my code. This is so that users at all levels can see more easily what is happening in the code, rather than creating functions to decrease repetition.

Here is the JSON file we will be working with.

[
    {
        "name": "Peppin",
        "begin": "751",
        "end": "786"
    },
    {
        "name": "Charlemagne",
        "begin": "786",
        "end": "814"
    },
    {
        "name": "Louis",
        "begin": "814",
        "end": "841"
    }
]

import json
filename = "./data/kings.json"
def Choices():
    print ("K I N G S")
    print ("Data Management System")
    print ("(1) View Data")
    print ("(2) Add Data")
    print ("(3) Delete Data")
    print ("(4) Edit Data")
    print ("(5) Exit")

The first thing we will do is import json (line 1). Next, we create an object that is our filename, so that we don’t have to write it throughout our script (line 2).

The Choices function creates the initial display when a user runs the program. They are prompted to select between viewing, adding, deleting, and editing the JSON data.

def view_data():
    with open (filename, "r") as f:
        temp = json.load(f)
        i = 0
        for entry in temp:
            name = entry["name"]
            begin = entry["begin"]
            end = entry["end"]
            print (f"Index Number {i}")
            print (f"Name of King  : {name}")
            print (f"Begin of Reign: {begin}")
            print (f"End of Reign  : {end}")
            print ("\n\n")
            i=i+1

Our view data function allows a user to view existing data. Most users do not enjoy reading JSON files and structure. Why this is the case, I cannot say. I find JSON to be beautiful. A rant for another day. In the above code, we define our function (line 12), open our json file (line 13), and create an object of that data (line 14). We create a counter with the object “i” (line 15) and open a for loop to iterate across our JSON data (line 16). For each entry, we create objects for the respective king’s name, begin (start of reign), and end (end of reign) (lines 17-19). We then print off an index number so that the user can see what index that entry has in the JSON list and the relevant data (lines 20-24). We then increase our counter for the next loop (line 25).

Next, we want the user to be able to add data to the JSON file. We do this in the code below.

def add_data():
    item_data = {}
    with open (filename, "r") as f:
        temp = json.load(f)
    item_data["name"] = input("Name of King: ")
    item_data["begin"] = input("Begin of Reign: ")
    item_data["end"] = input("End of Reign: ")
    temp.append(item_data)
    with open (filename, "w") as f:
        json.dump(temp, f, indent=4)

In line 27 we create the function. We then create an empty dictionary which we will append to our existing JSON database at the end of the function (line 28). We again open the JSON file and create an object of its contents (lines 29 and 30). Next, we prompt the user three times for the relevant data and create our dictionary: name, begin, and end (lines 31-33). In line 34, we append that dictionary to the existing JSON contents (line 34). In lines 35-36, we open our JSON database file and overwrite the existing contents with the new data that contains our user’s input.

Now that a user can add data, they also may want to be able to delete something. We can do that with the function below.

def delete_data():
    view_data()
    new_data = []
    with open (filename, "r") as f:
        temp = json.load(f)
        data_length = len(temp)-1
    print ("Which index number would you like to delete?")
    delete_option = input(f"Select a number 0-{data_length}")
    i=0
    for entry in temp:
        if i == int(delete_option):
            pass
            i=i+1
        else:
            new_data.append(entry)
            i=i+1
    with open (filename, "w") as f:
        json.dump(new_data, f, indent=4)

In line 39, we create the function. We then call view_data(), which allows us to see all the existing data (line 40). In line 42, we create an object, new_data, where we will store our JSON data without the item deleted. In lines 42-43, we open our JSON file, read its contents, and create an object. In line 44, we create an object that is the length of the list – 1 so that we can tell the user the range to select from in line 36 below.

We then prompt the user to select an index to delete (lines 45-46), save that object so that as we loop through our JSON file (line 48), we can do something different when we get to that index (line 49). Here, we pass the data. If the counter (i) does not equal the deleted option, then the function appends the JSON data to the new_data list. Essentially, this skips over the data that the user wants to delete and saves all other data. In lines 55-56, the function now opens and saves that new data into the same JSON database.

Sometimes a user may not want to delete an entry. They may want to simply edit one. In the function below, we give them that ability.

def edit_data():
    view_data()
    new_data = []
    with open (filename, "r") as f:
        temp = json.load(f)
        data_length = len(temp)-1
    print ("Which index number would you like to delete?")
    edit_option = input(f"Select a number 0-{data_length}")
    i=0
    for entry in temp:
        if i == int(edit_option):
            name = entry["name"]
            begin = entry["begin"]
            end = entry["end"]
            print (f"Current Name of King  : {name}")
            name = input("What would you like the new name to be? ")
            print (f"Current Begin of Reign: {begin}")
            begin = input("What would you like the new begin to be? ")
            print (f"Current End of Reign  : {end}")
            end = input("What would you like the new end to be? ")
            new_data.append({"name": name, "begin": begin, "end": end})
            i=i+1
        else:
            new_data.append(entry)
            i=i+1
    with open (filename, "w") as f:
        json.dump(new_data, f, indent=4)

This function does something very similar to delete until line 70. In lines 70-79, rather than skip the data, the user is told what the existing data is and then prompted to change it. The result is then appended to the new_data list, rather than the old data (line 79).

Finally, the above program will not work in a terminal without a while loop to keep it open.

while True:
    Choices()
    choice = input("\nEnter Number: ")
    if choice == "1":
        view_data()
    elif choice == "2":
        add_data()
    elif choice == "3":
        delete_data()
    elif choice == "4":
        edit_data()
    elif choice == "5":
        break
    else:
        print ("You did not select a number, please read more carefully.")

The above code opens the while loop (line 89) and continues to give the user a series of choices. Unless the user selects 5, which is exit, the loop will continue. Here is all the code combined and the three videos on Youtube that explain it, step-by-step.

import json
filename = "./data/kings.json"
def Choices():
    print ("K I N G S")
    print ("Data Management System")
    print ("(1) View Data")
    print ("(2) Add Data")
    print ("(3) Delete Data")
    print ("(4) Edit Data")
    print ("(5) Exit")

def view_data():
    with open (filename, "r") as f:
        temp = json.load(f)
        i = 0
        for entry in temp:
            name = entry["name"]
            begin = entry["begin"]
            end = entry["end"]
            print (f"Index Number {i}")
            print (f"Name of King  : {name}")
            print (f"Begin of Reign: {begin}")
            print (f"End of Reign  : {end}")
            print ("\n\n")
            i=i+1

def add_data():
    item_data = {}
    with open (filename, "r") as f:
        temp = json.load(f)
    item_data["name"] = input("Name of King: ")
    item_data["begin"] = input("Begin of Reign: ")
    item_data["end"] = input("End of Reign: ")
    temp.append(item_data)
    with open (filename, "w") as f:
        json.dump(temp, f, indent=4)


def delete_data():
    view_data()
    new_data = []
    with open (filename, "r") as f:
        temp = json.load(f)
        data_length = len(temp)-1
    print ("Which index number would you like to delete?")
    delete_option = input(f"Select a number 0-{data_length}")
    i=0
    for entry in temp:
        if i == int(delete_option):
            pass
            i=i+1
        else:
            new_data.append(entry)
            i=i+1
    with open (filename, "w") as f:
        json.dump(new_data, f, indent=4)


def edit_data():
    view_data()
    new_data = []
    with open (filename, "r") as f:
        temp = json.load(f)
        data_length = len(temp)-1
    print ("Which index number would you like to delete?")
    edit_option = input(f"Select a number 0-{data_length}")
    i=0
    for entry in temp:
        if i == int(edit_option):
            name = entry["name"]
            begin = entry["begin"]
            end = entry["end"]
            print (f"Current Name of King  : {name}")
            name = input("What would you like the new name to be? ")
            print (f"Current Begin of Reign: {begin}")
            begin = input("What would you like the new begin to be? ")
            print (f"Current End of Reign  : {end}")
            end = input("What would you like the new end to be? ")
            new_data.append({"name": name, "begin": begin, "end": end})
            i=i+1
        else:
            new_data.append(entry)
            i=i+1
    with open (filename, "w") as f:
        json.dump(new_data, f, indent=4)



while True:
    Choices()
    choice = input("\nEnter Number: ")
    if choice == "1":
        view_data()
    elif choice == "2":
        add_data()
    elif choice == "3":
        delete_data()
    elif choice == "4":
        edit_data()
    elif choice == "5":
        break
    else:
        print ("You did not select a number, please read more carefully.")

Python and PyVis for Data Visualization

I’ve recently been working with PyVis for my digital history project on Alcuin’s letters. PyVis is a Python module that reads in network data and then outputs a dynamic network graph that is coded in HTML, CSS, and Javascript. The outputted HTML file is based around the VIS.js library, a powerful library for producing Javascript graphs.

I have found this to be a nice alternative to NetworkX and Matplotlib because it allows digital humanists to create dynamic graphs, or graphs that allow users to interact with the graph. While NetworkX and Matplotlib are certainly useful and have their place in digital humanities projects, Matplotlib’s data visualizations have their limits. Javascript-based libraries provide a way around this.

In addition to this, the graphs produced are in html, which makes for easy web implementation. I can, for example, embed any graph in a website using that html code. See the image below:





In the image above, you can see ten of Alcuin’s letters (represented by blue ellipses) and the manuscripts that contain those letters (represented by purple triangles). These nodes in the graph are movable. The user can, therefore, interact with that data in a browser.

I have started a Youtube tutorial series on PyVis. For those of you who are interested in following along, here is the requisite JSON file of Alcuin’s letters: alcuin_letters.json