introduction
JSON is a common data format in data processing and analysis. Neo4j is a high-performance graph database that can store and query complex network relationships. This article will analyze a piece of Python code to introduce in detail how to process JSON data and import it into the Neo4j database.
Code structure overview
First, let’s take a look at the overall structure of the code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Time :2022/9/13 10:03
# @File :handler_person_data.py
# @Description: Processing json data
import json
import os
from common import constant
from common.conn_neo4j import ConnNeo4j
# Get the path to the data filedata_path = (constant.DATA_DIR, "")
# Read the contents of the data filedata = (open(data_path, 'r', encoding='utf-8'))
print("Number of characters:", len(data))
# Connect to Neo4j serverneo4j = ConnNeo4j()
# traverse datafor item in data:
item['name'] = item['Chinese name']
# Graduated school = []
if 'Graduated' in ():
school = item['Graduated']
('Graduated')
# work works = []
if 'work' in ():
works = item['work']
('work')
# Related People relate_persons = {}
if 'Related Characters' in ():
relate_persons = item['Related Characters']
('Related Characters')
print(item)
# Create character nodes neo4j.create_node("figure", item)
# Create school nodes, relationships between characters and schools neo4j.create_node_relations("figure", item, "School", school, "Graduated", {'type': 'Graduated'}, False)
# Create a node of the work, the relationship between characters and works neo4j.create_node_relations("figure", item, "work", works, "creation", {'type': 'creation'}, False)
# Create relevant characters, social relationships for key in relate_persons.keys():
tmp_value = relate_persons[key]
tmp_rel_type = key
if key in ['son', 'daughter', 'Father', 'Mother']:
neo4j.create_node_relations("figure", item, "figure", tmp_value, tmp_rel_type, {'type': 'Parent-child'}, False)
elif key in ['grandson', 'granddaughter', 'grandfather', 'grandmother']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Grandma'}, False)
elif key in ['elder brother', 'younger sister', 'younger brother', 'elder sister']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Brothers and sisters'}, False)
elif key in ['husband', 'wife']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'couple'}, False)
elif key in ['son in law', 'Daughter-in-law']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Son-in-law'}, False)
elif key in ['student', 'teacher']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Teacher and Student'}, False)
else:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'other'}, False)
Detailed code explanation
1. Import the necessary libraries
import json import os from common import constant from common.conn_neo4j import ConnNeo4j
json: used to process data in JSON format.
os: used to process file paths.
constant: Constant imported from the common module may contain information such as data directory.
ConnNeo4j: Neo4j connection class imported from common.conn_neo4j module.
2. Define the data file path
# Get the path to the data filedata_path = (constant.DATA_DIR, "")
data_path: Points to the JSON file path that contains the data.
3. Read the content of JSON file
# Read the contents of the data filedata = (open(data_path, 'r', encoding='utf-8'))
print("Number of characters:", len(data))
Use the() function to read the contents of the JSON file and store it in a data variable.
Print out the number of characters in the data.
4. Connect to Neo4j server
# Connect to Neo4j serverneo4j = ConnNeo4j()
Create a ConnNeo4j object to connect to the Neo4j database.
5. Traverse the data and process it
# traverse datafor item in data:
item['name'] = item['Chinese name']
# Graduated school = []
if 'Graduated' in ():
school = item['Graduated']
('Graduated')
# work works = []
if 'work' in ():
works = item['work']
('work')
# Related People relate_persons = {}
if 'Related Characters' in ():
relate_persons = item['Related Characters']
('Related Characters')
print(item)
# Create character nodes neo4j.create_node("figure", item)
# Create school nodes, relationships between characters and schools neo4j.create_node_relations("figure", item, "School", school, "Graduated", {'type': 'Graduated'}, False)
# Create a node of the work, the relationship between characters and works neo4j.create_node_relations("figure", item, "work", works, "creation", {'type': 'creation'}, False)
# Create relevant characters, social relationships for key in relate_persons.keys():
tmp_value = relate_persons[key]
tmp_rel_type = key
if key in ['son', 'daughter', 'Father', 'Mother']:
neo4j.create_node_relations("figure", item, "figure", tmp_value, tmp_rel_type, {'type': 'Parent-child'}, False)
elif key in ['grandson', 'granddaughter', 'grandfather', 'grandmother']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Grandma'}, False)
elif key in ['elder brother', 'younger sister', 'younger brother', 'elder sister']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Brothers and sisters'}, False)
elif key in ['husband', 'wife']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'couple'}, False)
elif key in ['son in law', 'Daughter-in-law']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Son-in-law'}, False)
elif key in ['student', 'teacher']:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'Teacher and Student'}, False)
else:
neo4j.create_node_relations('figure', item, 'figure', tmp_value, tmp_rel_type, {'type': 'other'}, False)
Iterates over each JSON object in data.
Rename the Chinese name field to name.
Processes the Graduate, Work, and Related Character fields and removes them from the JSON object.
Print the processed JSON object.
Call neo4j.create_node() method to create a character node.
Call the neo4j.create_node_relations() method to create schools, works and related character nodes and establish corresponding relationships.
Summarize
With this code, we learned how to extract data from a JSON file and import it into a Neo4j database. This process includes reading JSON files, processing data, creating nodes and relationships. Hope this article helps you understand how to process JSON data and import Neo4j databases.
This is the article about Python processing JSON data and importing Neo4j database. For more information about Python processing JSON data, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!