Neptune SPARQL QA Chain

Обновлено 24 мая 2024

This notebook shows use of LLM to query RDF graph in Amazon Neptune. This code uses a NeptuneRdfGraph class that connects with the Neptune database and loads it's schema. The NeptuneSparqlQAChain is used to connect the graph and LLM to ask natural language questions.

Requirements for running this notebook:

• Neptune 1.2.x cluster accessible from this notebook
• Kernel with Python 3.9 or higher
• For Bedrock access, ensure IAM role has this policy

{
        "Action": [
            "bedrock:ListFoundationModels",
            "bedrock:InvokeModel"
        ],
        "Resource": "*",
        "Effect": "Allow"
}

• S3 bucket for staging sample data, bucket should be in same account/region as Neptune.

Seed W3C organizational data

W3C org ontology plus some instances.

You will need an S3 bucket in the same region and account. Set STAGE_BUCKET to name of that bucket.

STAGE_BUCKET = "<bucket-name>"

%%bash  -s "$STAGE_BUCKET"

rm -rf data
mkdir -p data
cd data
echo getting org ontology and sample org instances
wget http://www.w3.org/ns/org.ttl 
wget https://raw.githubusercontent.com/aws-samples/amazon-neptune-ontology-example-blog/main/data/example_org.ttl 

echo Copying org ttl to S3
aws s3 cp org.ttl s3://$1/org.ttl
aws s3 cp example_org.ttl s3://$1/example_org.ttl

Bulk-load the org ttl - both ontology and instances

%load -s s3://{STAGE_BUCKET} -f turtle --store-to loadres --run

%load_status {loadres['payload']['loadId']} --errors --details

Setup Chain

EXAMPLES = """

<question>
Find organizations.
</question>

<sparql>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 
PREFIX org: <http://www.w3.org/ns/org#> 

select ?org ?orgName where {{
    ?org rdfs:label ?orgName .
}} 
</sparql>

<question>
Find sites of an organization
</question>

<sparql>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 
PREFIX org: <http://www.w3.org/ns/org#> 

select ?org ?orgName ?siteName where {{
    ?org rdfs:label ?orgName .
    ?org org:hasSite/rdfs:label ?siteName . 
}} 
</sparql>

<question>
Find suborganizations of an organization
</question>

<sparql>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 
PREFIX org: <http://www.w3.org/ns/org#> 

select ?org ?orgName ?subName where {{
    ?org rdfs:label ?orgName .
    ?org org:hasSubOrganization/rdfs:label ?subName  .
}} 
</sparql>

<question>
Find organizational units of an organization
</question>

<sparql>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> 
PREFIX org: <http://www.w3.org/ns/org#> 

select ?org ?orgName ?unitName where {{
    ?org rdfs:label ?orgName .
    ?org org:hasUnit/rdfs:label ?unitName . 
}} 
</sparql>

<question>
Find members of an organization. Also find their manager, or the member they report to.
</question>

<sparql>
PREFIX org: <http://www.w3.org/ns/org#> 
PREFIX foaf: <http://xmlns.com/foaf/0.1/> 

select * where {{
    ?person rdf:type foaf:Person .
    ?person  org:memberOf ?org .
    OPTIONAL {{ ?person foaf:firstName ?firstName . }}
    OPTIONAL {{ ?person foaf:family_name ?lastName . }}
    OPTIONAL {{ ?person  org:reportsTo ??manager }} .
}}
</sparql>


<question>
Find change events, such as mergers and acquisitions, of an organization
</question>

<sparql>
PREFIX org: <http://www.w3.org/ns/org#> 

select ?event ?prop ?obj where {{
    ?org rdfs:label ?orgName .
    ?event rdf:type org:ChangeEvent .
    ?event org:originalOrganization ?origOrg .
    ?event org:resultingOrganization ?resultingOrg .
}}
</sparql>

"""

import boto3
from langchain.chains.graph_qa.neptune_sparql import NeptuneSparqlQAChain
from langchain_community.chat_models import BedrockChat
from langchain_community.graphs import NeptuneRdfGraph

host = "<neptune-host>"
port = "<neptune-port>"
region = "us-east-1"  # specify region

graph = NeptuneRdfGraph(
    host=host, port=port, use_iam_auth=True, region_name=region, hide_comments=True
)

schema_elements = graph.get_schema_elements
# Optionally, you can update the schema_elements, and
# load the schema from the pruned elements.
graph.load_from_schema_elements(schema_elements)

bedrock_client = boto3.client("bedrock-runtime")
llm = BedrockChat(model_id="anthropic.claude-v2", client=bedrock_client)

chain = NeptuneSparqlQAChain.from_llm(
    llm=llm,
    graph=graph,
    examples=EXAMPLES,
    verbose=True,
    top_K=10,
    return_intermediate_steps=True,
    return_direct=False,
)

Ask questions

Depends on the data we ingested above

chain.invoke("""How many organizations are in the graph""")

chain.invoke("""Are there any mergers or acquisitions""")

chain.invoke("""Find organizations""")

chain.invoke("""Find sites of MegaSystems or MegaFinancial""")

chain.invoke("""Find a member who is manager of one or more members.""")

chain.invoke("""Find five members and who their manager is.""")

chain.invoke(
    """Find org units or suborganizations of The Mega Group. What are the sites of those units?"""
)