Json parsing using python
JSON (JavaScript Object Notation) is a text-based data format used for exchanging and storing data between web applications. It simplifies the data transmission process between different programming languages and platforms.
The JSON standard has become increasingly popular in recent years. It’s a simple and flexible way of representing data that can be easily understood and parsed by both humans and machines. JSON consists of key-value pairs enclosed in curly braces, separated by a colon.
Python provides various tools, libraries and methods for parsing and manipulating JSON data, making it a popular choice for data analysts, web developers, and data scientists.
In this guide, we’ll explore the syntax and data types of JSON, as well as the Python libraries and methods used for parsing JSON data, including more advanced options like JMESPath and ChompJS, which are very useful for web scraping data.
Reading JSON
One of the most common tasks when working with JSON data is to read its contents. Python provides several built-in libraries for reading JSON from files, APIs, and web applications. To read JSON data, you can use the built-in json module (JSON Encoder and Decoder) in Python.
The json module provides two methods, loads and load, that allow you to parse JSON strings and JSON files, respectively, to convert JSON into Python objects such as lists and dictionaries. Next is an example on how to convert JSON string to a Python object with the loads method.
import json json_input = ‘< "make": "Tesla", "model": "Model 3", "year": 2022, "color": "Red" >‘ json_data = json.loads(json_input) print(json_data) # Output:
Following, we display an example using the load method. Given a JSON file:
How to Parse JSON in Python
Invicti Web Application Security Scanner – the only solution that delivers automatic verification of vulnerabilities with Proof-Based Scanning™.
JSON is a popular format for data exchange. Python ships with a built-in JSON module to parse and work with JSON data. And this tutorial will teach you all about working with JSON in Python.
By the end of this tutorial, you’ll have learned:
- the basics of JSON,
- how to parse and create JSON strings in Python, and
- how to read from and write to JSON files in Python.
What is JSON?
JSON stands for J ava S cript O bject N otation, and it’s a text-based format for data interchange. Though JSON is initially inspired by JavaScript objects, almost all programming languages support working with JSON.
If you’ve ever worked with APIs or read through configuration files—you’d likely have run into JSON.
📑 You send and receive data in JSON when querying APIs. And JSON is also widely used in client-server communication in software applications. In addition, you can use JSON for general-purpose data storage as well.
The format of JSON is very similar to that of a Python dictionary. Dictionaries are powerful built-in data structures in Python that store data in key-value pairs.
Before we go any further, here are a few points worth noting:
- In Python, a JSON object is stored as a dictionary.
- An array in JSON is stored as a Python list.
- In JSON, the Boolean values are denoted as true and false . In Python, these are converted to the Booleans True and False .
For more details on the data types that are translated from JSON to Python, read the docs here.
As the json module is part of the Python standard library, you don’t have to install it. You can import into your current directory, like this:
How to Load a JSON String in Python
The general syntax to load a JSON string in Python is:
- is the Python dictionary to which you’d like to load the JSON string,
- is any valid JSON string.
This loads the into the Python dictionary .
Let’s code an example. Here json_str is a JSON string.
And the code snippet below shows how you can load the JSON string json_str into a Python dictionary using the loads() method. You can use the built-in type() function to verify that py_dict is a Python dictionary.
py_dict = json.loads(json_str) type(py_dict) # Output: dict print(py_dict) # Output , ]>As shown in the above code, all fields in the JSON string are key-value pairs in py_dict .
How to Create JSON Strings in Python
Let’s suppose you have a Python dictionary. So how do you create a JSON string from it?
You can do it using the dumps() method with this syntax:
- is the Python dictionary from which you’d like to create the JSON string,
- is the resultant JSON string.
So the dumps() method dumps into a JSON string .
To our existing Python dictionary py_dict . let’s add a new key «movies» . You can do it as shown in the following code snippet:
Now, let’s dump the modified dictionary to a new JSON string json_str2 using the dumps() method.
json_str2 = json.dumps(py_dict) print(json_str2) # Output , ], "movies": []> As you can see in the above example, the output JSON string is difficult to read through without proper formatting. You can use the optional parameter indent to add indentation.
And you can do this by setting indent to an integer like 2, as shown below:
json_str2 = json.dumps(py_dict, indent = 2) print(json_str2) # Output < "books": [ < "title": "The Wind in the Willows", "author": "Kenneth Grahame", "year": "1908" >, < "title": "To the Lighthouse", "author": "Virginia Woolf", "year": "1927" >], "movies": [ < "title": "The Imitation Game", "year": "2014", "lang": "en", "watched": true >] >Observe how the output has been formatted with indentation, and it’s easy to follow through.
Note: 💡 If you want the keys to be sorted in alphabetical order, you can set the sort_keys parameter to True .
As you can see in the code snippet below, the keys have now been sorted in alphabetical order.
json_str2 = json.dumps(py_dict, indent = 2, sort_keys=True) print(json_str2) # Output < "books": [ < "author": "Kenneth Grahame", "title": "The Wind in the Willows", "year": "1908" >, < "author": "Virginia Woolf", "title": "To the Lighthouse", "year": "1927" >], "movies": [ < "lang": "en", "title": "The Imitation Game", "watched": true, "year": "2014" >]And the keys now appear in alphabetical order: «author» , «title» and «year» .
So far, you’ve learned how to work with JSON strings in Python. In the next section, you’ll learn how to work with JSON files.
How to Read a JSON File in Python
To read a JSON file in Python, use the following syntax:
json.load() # where is any valid JSON file.Notice how we use the load() method and not the loads() method. loads() loads a JSON string, while load() loads a JSON file.
You should consider using context managers when working with files in Python. You can also try to read files as follows, without using context manager:
my_file = open('students.json','r') contents = my_file.read() print(contents) file.close()If you don’t close the file, there can be a potential wastage of resources.
However, when working with context managers, the files are automatically closed once the file operations are complete.
And you can use context manager to read files, as shown below:
with open('students.json','r') as file: data = json.load(file) print(data) # Output , ]>As you’re reading from a file, specify the mode as read—indicated by ‘r’ in the above code.
Note: In order to navigate easily through the current directory, please ensure that the JSON file is in the same folder as main.py , as shown in the image below. If your JSON file is in a different folder be sure to specify the path to the file.
In the next section, you’ll learn how to write to a JSON file.✍
How to Write to a JSON File in Python
To write to an existing JSON file or to create a new JSON file, use the dump() method as shown:
json.dump(,) # where is a Python dictionary # and is the JSON file So the above syntax dumps the dictionary into the JSON file .
In the previous section, we had the dictionary py_dict . Now let’s dump that into a new JSON file. And let’s name it new_file.json .
And the following code cell shows how you can use the dump() function:
with open('new_file.json','w') as file: json.dump(py_dict,file)Note: Opening a file in the write mode ( w ) overwrites the content if the file exists. If the file doesn’t exist, the file is created.
After executing the above code cell, you’ll see that a new JSON file has been created in the current working directory. And you can go ahead and examine the contents of the JSON file.
When writing to files, the key goal is data storage. And if you’d like to preserve formatting, you can also use the indent and sort_keys parameters.
Conclusion
⏲ It’s time for a quick summary.
In this tutorial, you have learned:
- the basics of using JSON,
- how to use the loads() and load() methods to read JSON string and JSON files respectively,
- how to use the dumps() and dump() methods to dump Python dictionaries into JSON strings and JSON files respectively.
I hope you found this tutorial helpful. Happy learning!