Core Concepts

Understanding how FOON works under the hood.

The Transformation Pipeline

Every FOON transformation follows this pipeline:

Input JSON → AI Mapping Plan → Deterministic Transform → Schema Validation → Output

1. Input Analysis

FOON analyzes the input JSON structure, identifying:

  • Field names and their semantic meaning
  • Data types and formats
  • Nested structures and arrays

2. Mapping Plan Generation

The AI generates a mapping plan that describes how to transform input to output:

interface MappingPlan {
  assignments: Assignment[];
}

interface Assignment {
  targetPath: string;     // JSONPath to target field
  sourcePath?: string;    // JSONPath to source field
  operation: string;      // Operation type
  confidence: number;     // 0-1 confidence score
}

3. Deterministic Execution

The mapping plan executes deterministically. This means:

  • Same input + same plan = same output, always
  • No AI randomness during transformation
  • Predictable, testable behavior

4. Schema Validation

The output is validated against your JSON Schema. If validation fails:

  • result.ok is false
  • result.error.category is 'VALIDATION_ERROR'
  • Invalid data never silently passes through

Confidence Scores

Each mapping step includes a confidence score (0-1):

ScoreMeaning
0.95+High confidence - semantic match is clear
0.80-0.94Good confidence - likely correct but verify
0.60-0.79Moderate - might need review
< 0.60Low - consider manual mapping

You can set a minimum confidence threshold:

import { transform, OpenAIProvider } from 'foon-sdk';

const result = await transform(input, {
  schema: mySchema,
  provider: new OpenAIProvider({
    apiKey: process.env.OPENAI_API_KEY,
    model: 'gpt-5-nano'
  }),
  confidenceThreshold: 0.85  // Reject mappings below this
});

if (!result.ok && result.error?.category === 'CONFIDENCE_TOO_LOW') {
  console.log('Rejected assignments:', result.trace.execution.assignmentsRejected);
}

Mapping Operations

Direct

Simple field copy with optional type coercion:

source.email_addr → target.email

Split

Split a single field into multiple target fields:

source.fullName → target.name.given + target.name.family

Merge

Combine multiple source fields into one:

source.firstName + source.lastName → target.fullName

Transform

Apply a transformation function:

source.timestamp (unix) → target.createdAt (ISO string)

Default

Set a constant or default value:

(constant: "Patient") → target.resourceType

Caching Strategy

FOON caches mapping plans to avoid redundant AI calls:

Using Cache

import { transform, OpenAIProvider, LRUCache } from 'foon-sdk';

const cache = new LRUCache({ max: 100, ttl: 3600000 });

const result = await transform(input, {
  schema,
  provider: new OpenAIProvider({
    apiKey: process.env.OPENAI_API_KEY,
    model: 'gpt-5-nano'
  }),
  cache
});

console.log('Cache hit:', result.trace.cache.hit);

Cache Keys

Plans are cached by a hash of:

  • Input structure (field names, nesting, types)
  • Target schema

The actual values don't affect the cache key, only structure matters.

Cache Benefits

  • First request: AI generates plan (~500-2000ms)
  • Subsequent requests: Use cached plan (~5-50ms)

Error Handling

FOON returns a result object with success status:

const result = await transform(input, { schema, provider });

if (!result.ok) {
  switch (result.error?.category) {
    case 'SCHEMA_LOAD_ERROR':
      // Invalid schema
      break;
    case 'PROVIDER_ERROR':
      // LLM API error
      break;
    case 'MAPPING_PLAN_PARSE_ERROR':
      // LLM returned invalid mapping plan
      break;
    case 'CONFIDENCE_TOO_LOW':
      // Mapping confidence below threshold
      break;
    case 'EXECUTION_ERROR':
      // Error applying mapping plan
      break;
    case 'VALIDATION_ERROR':
      // Output doesn't match schema
      break;
    case 'SECURITY_LIMIT_EXCEEDED':
      // Input exceeded security limits
      break;
  }
}

Trace Report

Every transformation includes a detailed trace:

const result = await transform(input, { schema, provider });

console.log(result.trace.traceId);           // Unique trace ID
console.log(result.trace.timings);           // Timing breakdown
console.log(result.trace.mappingPlan);       // The mapping plan used
console.log(result.trace.execution);         // Execution details
console.log(result.trace.validation);        // Validation results
console.log(result.trace.confidenceSummary); // Confidence statistics

Next Steps