Skip to content

Core Concepts

This page explains the key concepts and architecture of QueryLeaf to help you understand how it works and how to use it effectively.

Architecture Overview

QueryLeaf follows a modular architecture with three main components:

  1. SqlParser: Converts SQL text into an abstract syntax tree (AST)
  2. SqlCompiler: Transforms the AST into MongoDB commands
  3. CommandExecutor: Executes the commands against a MongoDB database

The flow of a query through the system is:

SQL Query → Parser → AST → Compiler → MongoDB Commands → Executor → Results

1. SQL Parser

The SQL Parser component takes raw SQL text and converts it into an abstract syntax tree (AST) using the node-sql-parser library. QueryLeaf extends the standard PostgreSQL dialect with additional support for:

  • Nested field access (e.g., address.city)
  • Array element access (e.g., items[0].name)

The parser uses preprocessing and postprocessing to handle these extensions.

2. SQL Compiler

The SQL Compiler takes the AST from the parser and converts it into MongoDB commands. It handles:

  • Mapping SQL operations to MongoDB operations (SELECT → find, INSERT → insertMany, etc.)
  • Converting SQL WHERE clauses to MongoDB query filters
  • Transforming JOINs into MongoDB $lookup aggregation stages
  • Converting GROUP BY clauses to MongoDB aggregation pipelines

3. Command Executor

The Command Executor takes the MongoDB commands generated by the compiler and executes them against a MongoDB database using your provided MongoDB client. It:

  • Executes the appropriate MongoDB methods based on the command type
  • Handles complex operations like aggregation pipelines
  • Returns the results in a consistent format

Key Objects

QueryLeaf

The main class that ties everything together. It provides a simple API to execute SQL queries against MongoDB:

const queryLeaf = new QueryLeaf(mongoClient, 'mydatabase');

// Execute a query and get all results as an array
const results = await queryLeaf.execute('SELECT * FROM users');

// Or use a cursor for more control and memory efficiency
// You can optionally specify the batch size to control memory usage
const cursor = await queryLeaf.executeCursor('SELECT * FROM users', { batchSize: 50 });
await cursor.forEach(user => {
  console.log(`Processing user: ${user.name}`);
});
await cursor.close();

DummyQueryLeaf

A special implementation of QueryLeaf that doesn't execute real MongoDB operations but logs them instead. Useful for testing and debugging:

const dummyLeaf = new DummyQueryLeaf('mydatabase');
await dummyLeaf.execute('SELECT * FROM users');
// [DUMMY MongoDB] FIND in mydatabase.users with filter: {}

// Cursor support works with DummyQueryLeaf too
const cursor = await dummyLeaf.executeCursor('SELECT * FROM users');
await cursor.forEach(user => {
  // Process mock data
});
await cursor.close();

Relationship Between SQL and MongoDB Concepts

QueryLeaf maps SQL concepts to MongoDB concepts:

SQL Concept MongoDB Equivalent
Database Database
Table Collection
Row Document
Column Field
JOIN $lookup
WHERE Query Filter
GROUP BY Aggregation ($group)
ORDER BY Sort
LIMIT Limit
SELECT Projection

Data Type Handling

QueryLeaf handles the conversion between SQL and MongoDB data types:

  • SQL strings → MongoDB strings
  • SQL numbers → MongoDB numbers
  • SQL dates → MongoDB dates
  • SQL NULL → MongoDB null
  • SQL booleans → MongoDB booleans

Naming Conventions

QueryLeaf uses specific naming conventions for mapping SQL to MongoDB:

  • SQL table names map directly to MongoDB collection names
  • Column names map directly to MongoDB field names
  • Nested fields use dot notation (e.g., address.city)
  • Array access uses dot notation with indices (e.g., items.0.name for SQL's items[0].name)

Execution Flow

QueryLeaf supports two main execution methods:

Standard Execution

When you call queryLeaf.execute(sqlQuery), the following happens:

  1. The SQL query is parsed into an AST
  2. The AST is compiled into MongoDB commands
  3. The commands are executed against the MongoDB database
  4. All results are loaded into memory and returned as an array

This is simple to use but can be memory-intensive for large result sets.

Cursor Execution

When you call queryLeaf.executeCursor(sqlQuery), the following happens:

  1. The SQL query is parsed into an AST
  2. The AST is compiled into MongoDB commands
  3. For SELECT queries, a MongoDB cursor is returned instead of loading all results
  4. You control how and when results are processed (streaming/batching)

This approach is more memory-efficient for large datasets and gives you more control.

If any step fails in either approach, an error is thrown with details about what went wrong.

Extending QueryLeaf

QueryLeaf is designed to be extensible. You can customize its behavior by:

  • Creating custom implementations of the interfaces
  • Extending the parser to support additional SQL syntax
  • Customizing the compiler to generate specialized MongoDB commands
  • Implementing a custom executor for specialized MongoDB operations

Next Steps

Now that you understand the core concepts of QueryLeaf, you can:

Copyright © 2023-2025 Beekeeper Studio, Inc