Knowledge Graph Embedder
Memory-efficient reasoning with on-demand subgraph loading
Overview
The Knowledge Graph Embedder (KGE) is an intermediary layer positioned between the modality-specific encoders and the universal 512d projection layer. It enables graph-aware encoding without requiring the full knowledge graph to reside in memory.
Problem Statement
Current architecture requires the full knowledge graph to be loaded in memory for GNN reasoning. This limits scalability and increases memory footprint.
Input → Encoder → Projection Layer → 512d Space → GNN → LLM
↑
Full graph in memory
(~800MB for 1M nodes)
Issues:
- Memory bound: Full graph must be loaded for GNN reasoning
- Scalability limit: Cannot grow beyond RAM capacity
- Latency: Full graph traversal for each query
- Power consumption: Unnecessary memory operations
Target Architecture
Input → Encoder → [KGE] → Projection Layer → 512d Space → GNN → LLM
↓
┌─────────────────────┐
│ On-Demand Subgraph │
│ Solid State Storage │
│ Incremental Compute │
└─────────────────────┘
Storage Layer
Technology: SQLite with FTS5 for full-text search, plus HNSW index for approximate nearest neighbor (ANN) search.
Schema
-- Core node storage
CREATE TABLE nodes (
node_id TEXT PRIMARY KEY,
embedding BLOB, -- 512d float32 vector
modality TEXT, -- 'text' | 'audio' | 'video' | 'ocr'
timestamp INTEGER,
source TEXT,
metadata JSON,
novelty_score REAL,
created_at INTEGER
);
-- Edge storage with type information
CREATE TABLE edges (
edge_id INTEGER PRIMARY KEY AUTOINCREMENT,
source_id TEXT,
target_id TEXT,
edge_type INTEGER, -- 0-10 (11 types)
weight REAL,
created_at INTEGER,
FOREIGN KEY (source_id) REFERENCES nodes(node_id),
FOREIGN KEY (target_id) REFERENCES nodes(node_id)
);
-- HNSW index for ANN search
CREATE VIRTUAL TABLE node_hnsw USING hnsw(
embedding,
metric='cosine',
m=16, -- connections per node
ef_construction=200 -- build-time accuracy
);
-- FTS5 for text search
CREATE VIRTUAL TABLE node_fts USING fts5(
node_id,
content,
content=nodes,
content_rowid=rowid
);
Storage Optimization
- Embeddings stored as BLOB (float32[512] = 2KB per node)
- 1M nodes = ~2GB on disk (vs 8GB in memory)
- NVMe SSD: ~3GB/s read speed (sub-millisecond access)
- Memory-mapped for hot access patterns
Subgraph Loader
Load only relevant subgraph for each query, using ANN search + graph traversal.
Algorithm
function loadRelevantSubgraph(query_embedding, max_nodes=100, hops=1):
# Step 1: Find k nearest nodes via ANN
seed_nodes = hnns_search(query_embedding, k=max_nodes)
# Step 2: Load 1-hop neighbors
subgraph_nodes = set(seed_nodes)
for node in seed_nodes:
neighbors = get_neighbors(node, edge_types=ALL)
subgraph_nodes.update(neighbors)
# Step 3: Load from disk (cache check first)
subgraph = []
for node_id in subgraph_nodes:
if node_id in cache:
subgraph.append(cache[node_id])
else:
node_data = load_from_disk(node_id)
cache[node_id] = node_data
subgraph.append(node_data)
# Step 4: Load edges within subgraph
edges = load_edges(subgraph_nodes)
return Subgraph(nodes=subgraph, edges=edges)
Cache Strategy
- LRU cache for hot subgraphs (10K nodes = ~20MB)
- Write-through cache for new nodes
- Background prefetch for frequently accessed regions
Performance Characteristics
Memory Usage
| Component | Current | With KGE | Reduction |
|---|---|---|---|
| Full graph in memory | 800 MB | 0 MB | 100% |
| Active subgraph | N/A | 20 MB | N/A |
| Cache (hot nodes) | N/A | 20 MB | N/A |
| HNSW index | N/A | 100 MB | N/A |
| Total | 800 MB | 140 MB | 82.5% |
Latency
| Operation | Current | With KGE | Change |
|---|---|---|---|
| ANN search (100 nodes) | N/A | 5 ms | New |
| Subgraph load (100 nodes) | N/A | 10 ms | New |
| GNN on subgraph | 50 ms | 15 ms | -70% |
| Total query | 100 ms | 30 ms | -70% |
Scalability
| Metric | Current | With KGE |
|---|---|---|
| Max nodes (8GB RAM) | 1M | 10M |
| Max nodes (32GB RAM) | 4M | 40M |
| Query latency (1M nodes) | 200 ms | 30 ms |
| Query latency (10M nodes) | OOM | 45 ms |