Graph Tools

Tools for building and querying the semantic knowledge graph.

generate_graph_edges

Generate edge candidates by finding semantically similar codes using pgvector KNN.

Parameters

Parameter	Type	Required	Default	Description
`k`	integer	No	`30`	Number of nearest neighbors per code
`min_similarity`	float	No	`0.65`	Minimum cosine similarity threshold
`level`	string	No	`"hts8"`	Embedding level to use
`variant`	string	No	`"full"`	Embedding variant to use

Response


{
  "run_id": "abc-123",
  "total_candidates": 142380,
  "codes_processed": 12769,
  "time_seconds": 312.5
}

Details

Uses PostgreSQL CROSS JOIN LATERAL with the <=> cosine distance operator:


SELECT sb.code AS source_code, nn.code AS target_code,
       1 - (sb.embedding <=> nn.embedding) AS similarity
FROM source_batch sb
CROSS JOIN LATERAL (
    SELECT e.code FROM hts_embeddings e
    WHERE e.code != sb.code
    ORDER BY e.embedding <=> sb.embedding
    LIMIT k
) nn
WHERE 1 - (sb.embedding <=> nn.embedding) >= min_similarity

Candidates are stored in hts_edge_candidates with source/target chapters and headings for efficient filtering.

classify_graph_edges

Classify edge candidates using an LLM into 10 relationship types. Async, resumable pipeline.

Parameters

Parameter	Type	Required	Default	Description
`concurrency`	integer	No	`200`	Maximum concurrent LLM calls
`batch_size`	integer	No	`500`	Database flush batch size

Response


{
  "run_id": "def-456",
  "classified": 142380,
  "errors": 12,
  "db_flushed": 142368,
  "time_seconds": 8234.1
}

Details

The pipeline:

Fetches unclassified candidates (LEFT JOIN to hts_semantic_edges for resume support)
Chunks candidates with configurable delay between chunks for rate-limit pacing
Sends each pair to GPT-5.4 Nano with a structured JSON schema
Validates the classification output
Batch-flushes results to hts_semantic_edges

Resumable: If interrupted, re-running picks up where it left off, only unclassified candidates are processed.

Classification Output Schema

Each classified edge includes:

Field	Type	Description
`relationship_type`	string	One of 10 types
`confidence`	float	0-1 classification confidence
`reasoning`	string	2 sentences, 25-30 words
`key_differentiator`	string	Max 8 words
`classification_clarity`	float	0-1 how clear-cut the classification was
`source_code_inclusionary`	list	Terms that define the source code
`source_code_exclusionary`	list	Terms that exclude from the source code
`target_code_inclusionary`	list	Terms that define the target code
`target_code_exclusionary`	list	Terms that exclude from the target code
`relationship_haiku`	string	5-7-5 syllable haiku
`bidirectional`	boolean	Whether the relationship is symmetric

get_graph_status

Get pipeline statistics including candidate counts, classification progress, and type breakdown.

Parameters

None.

Response


{
  "total_candidates": 142380,
  "total_classified": 142368,
  "by_type": {
    "material_affinity": { "count": 18420, "avg_confidence": 0.82 },
    "functional_similarity": { "count": 15230, "avg_confidence": 0.79 },
    "manufacturing_process": { "count": 12100, "avg_confidence": 0.76 },
    "end_use": { "count": 11890, "avg_confidence": 0.74 },
    "substitution": { "count": 8920, "avg_confidence": 0.71 },
    "component_assembly": { "count": 7650, "avg_confidence": 0.73 },
    "cross_category_bridge": { "count": 3210, "avg_confidence": 0.68 },
    "abstraction": { "count": 2890, "avg_confidence": 0.85 },
    "weak_association": { "count": 24560, "avg_confidence": 0.45 },
    "no_meaningful_relationship": { "count": 37498, "avg_confidence": 0.88 }
  },
  "avg_confidence": 0.69,
  "recent_runs": [...]
}

get_code_relationships

Query classified semantic edges for a specific HTS code.

Parameters

Parameter	Type	Required	Default	Description
`hts_code`	string	Yes	-	HTS code to query (any length)
`min_confidence`	float	No	`0.0`	Minimum confidence threshold
`relationship_types`	list[string]	No	All types	Filter to specific relationship types
`limit`	integer	No	`20`	Maximum results

Response


{
  "hts_code": "02013080",
  "edges": [
    {
      "id": 12345,
      "source_code": "02013080",
      "target_code": "02023080",
      "relationship_type": "material_affinity",
      "confidence": 0.92,
      "reasoning": "Both classify boneless beef cuts. Source is fresh/chilled while target is frozen, same animal origin.",
      "key_differentiator": "fresh vs frozen preservation",
      "bidirectional": true,
      "similarity": 0.89,
      "haiku": "Cattle on the range\nFresh and frozen sides of beef\nCold preserves the cut"
    }
  ],
  "count": 15
}

Examples

All relationships for a code:


{
  "hts_code": "0201.30.80"
}

High-confidence substitution relationships only:


{
  "hts_code": "0201.30.80",
  "min_confidence": 0.8,
  "relationship_types": ["substitution", "functional_similarity"]
}