Cipher Corpus Statistics and Evaluation Status

Generated 2026-10-26 · v0.6 · Download JSON

This report inventories the published v0.6 archive. The corrected v0.8 dataset has separate versioned statistics and verification notes.

Evaluation Status

No solver accuracy has been measured for this report. Earlier human, automated-solver and LLM percentages were unsupported estimates, not benchmark results, and have been removed. Difficulty labels are curriculum metadata, not calibrated success probabilities.

The downloadable “blind” split is public and includes answers. It is not confidential held-out data. Records labeled historical include teaching adaptations; the legacy verified flag does not establish source authenticity. See dataset scope and errata.

Record Counts by Difficulty

Per-Cipher Statistics (Top 50 by Record Count)

“Mixed” means records of that cipher type carry more than one difficulty label. The JSON report includes each difficulty distribution.

Collection Sizes

These counts describe different collections and evaluation methods; they do not compare solver accuracy or historical authenticity.

Citation

@dataset{clark_2026_23297920,
  author={Clark, Paul A.},
  title={Cipher Corpus v0.6: A Classical Cryptography Dataset},
  month={oct},
  year={2026},
  publisher={Zenodo},
  version={0.6},
  doi={10.5281/zenodo.23297920},
  url={https://doi.org/10.5281/zenodo.23297920}
}

@article{li2025cipherbank,
  title={CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges},
  author={Li, Yu and Pei, Qizhi and Sun, Mengyuan and Lin, Honglin and Ming, Chenlin and Gao, Xin and Wu, Jiang and He, Conghui and Wu, Lijun},
  journal={arXiv preprint arXiv:2504.19093},
  year={2025},
  url={https://arxiv.org/pdf/2504.19093}
}