SynSQL — Single‑cell

Single‑cell deep dive

What you will find here

This page presents a concrete example of how SynSQL handles large single‑cell datasets. We describe the test dataset, the structure of the resulting table, and the ingestion performance obtained during bulk loading and query performance.

Test cluster

The test cluster consists of two DELL R6515 servers, each with an AMD Epyc 7302P 16-core processor, 128 GB of RAM and 6 x 2 TB SSDs. This configuration is representative of a modest two‑node research cluster. Even at this scale, SynSQL comfortably handles multiple wide scientific tables.

Test dataset

The dataset used for this benchmark is a typical single‑cell matrix with exactly one million cells, 20 000 gene columns and an id. The raw data was generated from a C program in text format. It was converted into a wide table suitable for SynSQL. We did not include additional analytical columns such as clustering labels, embeddings (UMAP/PCA), or pseudotime as 100 more columns wouldn't make any difference.

Table structure

The single‑cell table contains one row per cell and thousands of columns. These are:

• 1 id key. A mandatory hash key value
• 20 000 Gene expression features

All values are stored using simple SQL types (INT and fixed‑point encoded integers). No specialized format is required, and the entire dataset fits into a single wide table.

Ingestion performance

SynSQL supports both high‑throughput bulk loading and standard SQL mutability (INSERT, UPDATE, DELETE). This combination allows fast ingestion of large datasets while preserving the flexibility of fine‑grained modifications.

Table creation

The table is created in 5 seconds, even with 20 001 columns. Column definitions are generated programmatically from the dataset schema.

Deployment

The time to deploy the table across the cluster was 117 seconds.

Bulk load throughput

Bulk loading achieves very high throughput. In our test, 92 GB of single‑cell data were ingested in 134 seconds, corresponding to an average rate of approximately 700 MB/s across the cluster.

Row creation

The time to create a single row from the command line tool is ~1.25 seconds average

Query performance

The following examples illustrate typical single‑cell queries executed on the dataset described further ahead. Each query selects a subset of cells based on simple gene‑level conditions. Execution times include filtering, projection, and result materialization.
The result of the last query shows that queries involving an OR clause on distant columns involve an assembly of all the rows requested (here, a million), which is expensive.

Single‑gene filter

SELECT id, gene_00000

      FROM single_cell

      WHERE gene_00000 < 100;

→ 10 854 rows in 1.227920 seconds.

Multi‑gene filter

SELECT id, gene_00005, gene_12345, gene_15000

      FROM single_cell

      WHERE gene_00005 < 500

      AND gene_12345 > 1000

      AND gene_15000 < 2000;

→ 8 193 rows in 6.720899 seconds.

Two‑gene low‑expression slice

SELECT id, gene_00002, gene_19999

      FROM single_cell

      WHERE gene_00002 < 100

      AND gene_19999 < 100;

→ 96 rows in 0.433283 seconds.

OR‑based multi‑gene filter

SELECT id, gene_00001, gene_10000, gene_19999

      FROM single_cell

      WHERE gene_00001 < 50

      OR gene_10000 < 50

      OR gene_19999 < 50;

→ 13 954 rows in 73.460834 seconds.