Cipher Corpus Benchmark Baselines

Generated 2026-10-11 · v0.6 · Download JSON

100,326

Total Records

Cipher Types

Languages

30194

Blind Split Records

Historical Records

Expected Solver Accuracy by Difficulty

Difficulty	Records	Human Expert	Automated Solver	LLM 3-Shot	LLM 0-Shot Challenge
beginner	15907	95-100%	95-100%	80-95%	60-80%
intermediate	29510	60-90%	60-90%	40-70%	20-50%
advanced	27908	20-60%	20-60%	10-40%	5-20%
expert	27001	5-30%	5-30%	2-15%	1-5%

Per-Cipher Statistics (Top 50 by Record Count)

Cipher Type	Family	Difficulty	Records	Avg plaintext symbols	LLM 3-Shot	LLM 0-Shot
caesar	substitution	beginner	5102	57	80-95%	60-80%
vigenere	polyalphabetic	intermediate	4979	58	40-70%	20-50%
affine	substitution	beginner	4635	60	80-95%	60-80%
monoalphabetic	substitution	beginner	3782	62	80-95%	60-80%
columnar_transposition	transposition	intermediate	3490	59	40-70%	20-50%
playfair	polygraphic	intermediate	3036	54	40-70%	20-50%
beaufort	polyalphabetic	intermediate	2815	59	40-70%	20-50%
enigma	machine	expert	2723	62	2-15%	1-5%
gronsfeld	polyalphabetic	intermediate	2538	63	40-70%	20-50%
alberti_disk	disk	expert	2141	63	2-15%	1-5%
rail_fence	transposition	intermediate	2125	57	40-70%	20-50%
bazeries	transposition	advanced	1970	62	10-40%	5-20%
autokey	polyalphabetic	intermediate	1910	58	40-70%	20-50%
scytale	transposition	intermediate	1860	61	40-70%	20-50%
bifid	fractionation	advanced	1853	54	10-40%	5-20%
confederate_vigenere	polyalphabetic	intermediate	1682	62	40-70%	20-50%
diana	polyalphabetic	intermediate	1681	62	40-70%	20-50%
porta	polyalphabetic	intermediate	1680	62	40-70%	20-50%
adfgx	fractionation	advanced	1552	57	10-40%	5-20%
trifid	fractionation	advanced	1530	61	10-40%	5-20%
fractionated_morse	fractionation	advanced	1450	61	10-40%	5-20%
hill	polygraphic	advanced	1450	61	10-40%	5-20%
adfgvx	fractionation	advanced	1391	62	10-40%	5-20%
chaocipher	machine	advanced	1391	62	10-40%	5-20%
double_transposition	transposition	advanced	1390	62	10-40%	5-20%
lorenz	machine	expert	1301	58	2-15%	1-5%
four_square	polygraphic	advanced	1284	57	10-40%	5-20%
stager_route	transposition	expert	1282	59	2-15%	1-5%
m209	machine	expert	1221	60	2-15%	1-5%
kryptos	polyalphabetic	advanced	1162	61	10-40%	5-20%
sigaba	machine	expert	1161	61	2-15%	1-5%
fialka	machine	expert	1161	61	2-15%	1-5%
kl7	machine	expert	1161	61	2-15%	1-5%
geheimschreiber	machine	expert	1161	61	2-15%	1-5%
kama_sutra	substitution	intermediate	1160	61	40-70%	20-50%
cardano_autokey	polyalphabetic	advanced	1160	61	10-40%	5-20%
red_type_a	machine	advanced	1160	61	10-40%	5-20%
typex	machine	expert	1160	61	2-15%	1-5%
wadsworth	polyalphabetic	advanced	1100	62	10-40%	5-20%
vic	substitution	advanced	1074	56	10-40%	5-20%
nihilist	fractionation	advanced	1072	56	10-40%	5-20%
two_square	polygraphic	advanced	1072	56	10-40%	5-20%
geez_monastic	substitution	advanced	930	60	10-40%	5-20%
purple	machine	expert	872	61	2-15%	1-5%
copiale	substitution	advanced	871	61	10-40%	5-20%
wheatstone	polygraphic	advanced	871	61	10-40%	5-20%
great_cipher	nomenclator	expert	871	61	2-15%	1-5%
jn25	codebook	expert	871	61	2-15%	1-5%
venona_pad_reuse	stream	expert	871	61	2-15%	1-5%
straddling_checkerboard	substitution	advanced	870	61	10-40%	5-20%

Comparison with CipherBank

Benchmark	Records	Algorithms	Historical	Blind Splits	Multilingual
CipherBank (Li et al., 2025)	2,358	9	No	No	No
Cipher Corpus v0.6	100,326	85	Yes (54)	Public labels (30194)	Yes (9 langs)

Citation

@dataset{clark_2026_23297920,
  author={Clark, Paul A.},
  title={Cipher Corpus v0.6: A Classical Cryptography Dataset},
  month={oct},
  year={2026},
  publisher={Zenodo},
  version={0.6},
  doi={10.5281/zenodo.23297920},
  url={https://doi.org/10.5281/zenodo.23297920}
}

@article{li2025cipherbank,
  title={CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenges},
  author={Li, Yu and Pei, Qizhi and Sun, Mengyuan and Lin, Honglin and Ming, Chenlin and Gao, Xin and Wu, Jiang and He, Conghui and Wu, Lijun},
  journal={arXiv preprint arXiv:2504.19093},
  year={2025},
  url={https://arxiv.org/pdf/2504.19093}
}