JSON (JavaScript Object Notation) is a widely used format for storing and exchanging data. It is used in web applications, APIs, and databases due to its lightweight and human-readable format. JSON was inspired by the programming language JavaScript, but is now supported by most programming languages. JSON structures data in key-value pairs, so it’s easy to work with across different platforms. JSON uses double quotes to enclose strings, and all the keys in JSON must be strings.

In Python, working with JSON files is easy because of Python’s built in package, json, which is included with Python as part of the standard library. The json module is a powerful tool for reading, parsing, and processing JSON data efficiently. JSON is one of the likely data formats you will encounter when using web scraping services like Crawlbase, which delivers scraped content as clean, structured JSON.

JSON is a string representation of data, which is converted into Python data structures (such as dictionaries) when read. This Python JSON file parsing tutorial will cover how to read JSON files in Python, load and parse JSON data, and working with its structures. You’ll also learn how to modify and write JSON data. Let’s get started!

Here’s a simple tutorial on how to read JSON files in Python:

Table of Contents

  1. What are JSON Files?
  2. Loading JSON Data in Python
  • Using json.load() to Read JSON from a File
  • Reading JSON from a String with json.loads()
  • Fetching and Parsing JSON from Web Responses
  1. Common Operations with JSON in Python
  • Accessing Nested Data
  • Modifying JSON Data
  • Writing JSON Data to a File with json.dump()
  1. Error Handling When Reading JSON Files
  2. Final Thoughts
  3. Frequently Asked Questions (FAQs)

What are JSON Files?

A JSON file (JavaScript Object Notation file) is a text file used to store and exchange data. It stores data in a structured and readable way, using key value pairs, where each key is associated with a value. This makes it easy for humans and machines to read and write data. JSON is used in web applications, APIs, and configurations because it’s lightweight and easy to use.

Here’s an example of a simple JSON file:

1
2
3
4
5
6
{
  "name": "Jane Smith",
  "age": 28,
  "isDeveloper": true,
  "languages": ["Python", "JavaScript", "SQL"]
}

In this example, the file contains information about a person named “Jane Smith” who is 28 and a developer. The languages key holds an array of programming languages she knows.

You can read, write and save a JSON file in Python. Let’s see how. If you’re working on data-heavy applications or building APIs, it’s often helpful to hire backend developer who understands how to manage JSON efficiently.

How to Load JSON Data from Python

To work with JSON in Python, you first need to import the json module, which is a built-in package. The json module provides several functions for reading, writing, and manipulating JSON data. When you load JSON data, it is converted into Python objects and Python values.

Loading JSON in Python is easy, thanks to the built-in json module. Whether the JSON is in a file or a string, Python has methods to load and parse it. In this section, we will cover two ways to load JSON: from a file and from a string.

How to Use json.load() to Read JSON from a File

When working with a JSON file in Python, you can use json.load() to load the data directly from the file. This method reads the file and parses it into a Python object (usually a dictionary).

Here’s an example of how to read a JSON file:

1
2
3
4
5
6
7
8
import json

# Open the JSON file and load its content
with open('data.json', 'r') as file:
    data = json.load(file)

# Print the parsed data
print(data)

In this code, We use open() function to open the file in read mode (‘r’) and then pass the file object to json.load() to read and parse the JSON into a Python dictionary. Then, you can access the data using the keys.

For example, if your data.json file contains the following:

1
2
3
4
5
{
  "name": "Alice",
  "age": 25,
  "isEmployed": true
}

The output will be:

1
{'name': 'Alice', 'age': 25, 'isEmployed': True}

How to Read JSON from a String with json.loads()

If the JSON is a string, you can use json.loads() to parse it. This is useful when working with JSON data retrieved from an API or other external source.

Here’s an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import json

# JSON string

json_string = '{"product": "Laptop", "price": 999, "inStock": true}'

# Parse the string into a Python dictionary

data = json.loads(json_string)

# Print the parsed data

print(data)

In this example, json.loads() takes the JSON string and turns it into a Python dictionary. You can then access the data as if it was from a file.

For example, the output will be:

1
{'product': 'Laptop', 'price': 999, 'inStock': True}

How to Fetch and Parse JSON from Web Responses

So far, we’ve covered reading JSON from strings and files. In real-world scenarios, JSON data often comes from live websites or APIs. While simple libraries like requests work for public APIs, some JSON endpoints are behind JavaScript or anti-bot measures. This is where Crawlbase Crawling API helps.

Crawlbase enables you to fetch web pages reliably, even if the content is dynamically rendered. Here’s how to use the Crawlbase Python library to get a JSON response and parse it using json.loads().

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from crawlbase import CrawlingAPI
import json

# Initialize Crawlbase with your token
crawling_api = CrawlingAPI({'token': 'YOUR_CRAWLBASE_TOKEN'})

def fetch_json(url):
    response = crawling_api.get(url)

    if response['headers']['pc_status'] == '200':
        try:
            return json.loads(response['body'].decode('utf-8'))
        except json.JSONDecodeError:
            print("Invalid JSON.")
            return None
    print(f"Failed to fetch. Status: {response['headers']['pc_status']}")
    return None

# Example usage
url = 'https://jsonplaceholder.typicode.com/users'
data = fetch_json(url)
if data:
    print(json.dumps(data, indent=4))

Tip: Crawlbase offers Normal Tokens for static sites and JS Tokens for dynamic content. Get 1,000 free requests when you sign up, no credit card is required.

Example Output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[
    {
        "id": 1,
        "name": "Leanne Graham",
        "username": "Bret",
        "email": "Sincere@april.biz",
        "address": {
            "street": "Kulas Light",
            "suite": "Apt. 556",
            "city": "Gwenborough",
            "zipcode": "92998-3874",
            "geo": {
                "lat": "-37.3159",
                "lng": "81.1496"
            }
        },
        "phone": "1-770-736-8031 x56442",
        "website": "hildegard.org",
        "company": {
            "name": "Romaguera-Crona",
            "catchPhrase": "Multi-layered client-server neural-net",
            "bs": "harness real-time e-markets"
        }
    },
    .... more
]

Next up we’ll cover the common operations you can do on JSON in Python.

Common Operations with JSON in Python

Once you have loaded JSON data into Python, it is typically represented as data structures such as python dictionaries, which allow you to access and modify each key value pair efficiently. You can do various things with it, like access nested data, modify it, and save the changes back to a file. Let’s go through these one by one with simple examples.

How to Access JSON Nested Data

JSON data often has nested structures like dictionaries within dictionaries or lists within lists. Accessing this nested data in Python is easy using key-value access or list indexing.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
import json

# Sample JSON data
json_data = '''
{
    "name": "Alice",
    "age": 25,
    "address": {
        "city": "New York",
        "zipcode": "10001"
    },
    "skills": ["Python", "Machine Learning"]
}
'''

# Parse the JSON data
data = json.loads(json_data)

# Accessing nested data
city = data['address']['city']
skill = data['skills'][0]

print(f"City: {city}")
print(f"First Skill: {skill}")

Here:

  • We first load the JSON string using json.loads().
  • We access nested data, such as the city inside the address dictionary and the first skill from the skills list.

How to Modify JSON Data

You can easily modify JSON data in Python after loading it. Modifications can include updating values, adding new data, or removing existing keys.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import json

# Sample JSON data
json_data = '''
{
    "name": "Alice",
    "age": 25,
    "address": {
        "city": "New York",
        "zipcode": "10001"
    }
}
'''

# Parse the JSON data
data = json.loads(json_data)

# Modify the age and add a new skill
data['age'] = 26
data['skills'] = ["Python", "Data Science"]

# Print the modified data
print(json.dumps(data, indent=4))

In this example:

  • We modify the age value from 25 to 26.
  • We add a new key skills with an array of values.
  • The json.dumps() function is used to print the modified JSON data in a readable format with indentation.

How to Write JSON Data to a File with json.dump()

After modifying JSON data, you might want to save it back to a file. You can create a new json file or update an existing one using the json.dump() function, which helps you write the data back to a file in JSON format.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import json

# Sample data to write to file
data = {
    "name": "Alice",
    "age": 26,
    "address": {
        "city": "New York",
        "zipcode": "10001"
    },
    "skills": ["Python", "Data Science"]
}

# Write JSON data to a file
with open('modified_data.json', 'w') as file:
    json.dump(data, file, indent=4)

print("JSON data successfully written to file.")

In this example:

  • We modify and organize the data in a Python dictionary.
  • The json.dump() method writes the data to a file named modified_data.json.
  • The indent=4 parameter makes the JSON file readable by adding indentation.

Learning these common operations (accessing nested data, modifying it, and saving it to a file) is very important for working with JSON files in Python. They allow you to manipulate and organize your data for many use cases.

Next up, we’ll cover error handling when reading JSON files so your programs don’t crash.

How to handle Errors When Reading JSON Files

When working with JSON data in Python, you need to handle errors that can happen when reading or parsing JSON files. Errors can occur due to many reasons such as invalid JSON syntax, incorrect file paths, or file encoding issues. Proper error handling will ensure your Python script runs smoothly and can recover from unexpected errors.

Let’s explore some common errors and how to handle them effectively in Python.

How to Handle JSON File Not Found Error

If the specified JSON file does not exist or the file path is incorrect, Python raises a FileNotFoundError. You can use a try-except block to catch this error and display a user-friendly message.

Example:

1
2
3
4
5
6
7
8
import json

try:
    # Attempt to open a non-existent file
    with open('data.json', 'r') as file:
        data = json.load(file)
except FileNotFoundError:
    print("Error: The file was not found. Please check the file path.")

In this code:

  • We attempt to read the data.json file.
  • If the file does not exist, the FileNotFoundError is caught, and a meaningful error message is printed.

How to Handle Invalid JSON Syntax

If the JSON file contains invalid syntax (e.g., missing commas, braces, or brackets), Python raises a json.JSONDecodeError. You can handle this error using a try-except block to prevent your program from crashing.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import json

invalid_json = '''
{
    "name": "Alice",
    "age": 25,
    "address": {
        "city": "New York"
    }  # Missing closing bracket
'''

try:
    # Attempt to parse invalid JSON
    data = json.loads(invalid_json)
except json.JSONDecodeError as e:
    print(f"Error: Invalid JSON format. {e}")

Here:

  • The invalid_json string is missing a closing bracket.
  • The json.JSONDecodeError is caught, and an error message specifying the issue is printed.

How to Handle Incorrect JSON File Encoding

Sometimes, the JSON file might be saved with an encoding different from UTF-8, which can cause decoding errors when reading the file. Python’s UnicodeDecodeError handles such cases, and you can specify the correct encoding while opening the file to avoid issues.

Example:

1
2
3
4
5
6
7
8
import json

try:
    # Open a file with a specific encoding
    with open('data.json', 'r', encoding='utf-8') as file:
        data = json.load(file)
except UnicodeDecodeError:
    print("Error: Unable to decode the file. Please check the file's encoding.")

In this code:

  • We specify encoding='utf-8' when reading the file.
  • If there is a problem with the file encoding, a UnicodeDecodeError is caught and an appropriate error message is displayed.

General Exception Handling

You can also use a general except block to catch any other unexpected errors that might occur when reading or working with JSON files.

Example:

1
2
3
4
5
6
7
8
import json

try:
    # Attempt to read JSON file
    with open('data.json', 'r') as file:
        data = json.load(file)
except Exception as e:
    print(f"An unexpected error occurred: {e}")

This code:

  • Uses a general Exception to catch any errors that don’t fall into specific categories.
  • Prints the error message to help identify the problem.

Error handling is an essential part of working with JSON files, as it helps you manage issues like missing files, incorrect formats, and encoding problems. By catching these errors early, you can ensure that your Python scripts run more smoothly and are easier to debug.

Collect Data with Crawlbase

Reading and working with JSON files in Python is a crucial skill for developers, especially when dealing with APIs, web applications, or data storage. Python allows you to work with json data that is stored in files or variables. The json module is a built in package, making it easy to handle json objects and data structures in your projects, whether you’re reading from a file, parsing a string, or modifying the data.

By learning how to load, manipulate, and write JSON data in Python, you can efficiently manage structured data in your projects. JSON’s flexibility and readability make it one of the most widely used formats today, and with Python’s tools, you can easily integrate it into any application.

If you’re working with web data that’s returned in JSON, such as APIs, AJAX-loaded content, or REST endpoints, tools like the Crawling API or Smart AI Proxy make it easy to fetch that data without getting blocked. Once you’ve collected the JSON response, you can load and parse it in Python using the method above to work with json objects and data stored in variables or files.

Want to test this with real JSON data? Sign up now to fetch web data and export it as JSON.

Frequently Asked Questions (FAQs)

Q. What is the difference between json.load() and json.loads() in Python?

json.load() is used to read and parse JSON data from a file, while json.loads() is used to parse JSON data from a string. The s in json.loads() stands for “string”, so it’s useful when you have JSON data as a string, not in a file.

How do I convert a JSON string to a Python object?

You can convert a JSON string to a Python object using json.loads(). This function parses the JSON string and returns a Python dictionary or list.

Example:

1
2
3
4
5
import json

json_string = '{"name": "John", "age": 30}'
data = json.loads(json_string)
print(data)

Q. How do I write JSON data to a file in Python?

To write JSON data to a file, use json.dump(). Open the file in write mode, then pass the Python object and file to json.dump() to store the JSON data in the file.

Example:

1
2
3
4
5
6
import json

data = {"name": "John", "age": 30}

with open('data.json', 'w') as file:
    json.dump(data, file)

Q. How do I handle errors when working with JSON in Python?

Common errors when working with JSON include invalid JSON format or incorrect file paths. To handle these, you can use Python’s try-except blocks to catch exceptions like json.JSONDecodeError or FileNotFoundError.

Example:

1
2
3
4
5
6
7
8
9
import json

try:
    with open('data.json', 'r') as file:
        data = json.load(file)
except FileNotFoundError:
    print("File not found!")
except json.JSONDecodeError:
    print("Invalid JSON format!")