Apiary

A public, tiny AI lab
Statuslive
Tokens0
Crawlers0 out ○Brainheuristic

Dataset

Every page the crawlers kept: relevant, quality 3 or better, and not a copy of anything already here. Exact copies are caught by SHA-256, near-copies by MinHash (Jaccard above 0.8).

Unique tokens10.84MPages1,907
Storagelocal volumePDFs50

By topic

TopicSharePagesTokens
defi████████████████████1914.23M
markets███████████·········6292.29M
ethereum██████··············2411.21M
infrastructure██████··············4051.20M
solana███·················131670.4K
bitcoin███·················80584.7K
governance██··················183520.9K
layer 2█···················47132.5K

Why pages were dropped

irrelevant3,225
thin507
robots505
low quality213
http 403146
near duplicate93
http 40475
not html59
duplicate57
http 42920
pdf too large3
http 4022
http 5002
http 4061

The ladder

VersionDepthParamsTokens neededProgress
v1436.70M91.75M██·················· 11.8%
v2660.95M152.37M█··················· 7.1%
v3892.27M230.69M█··················· 4.7%
v412185.60M463.99M···················· 2.3%
v516335.54M838.86M···················· 1.3%
v620560.99M1.40B···················· 0.773%
v7261.08B2.70B···················· 0.401%
v8301.58B3.95B···················· 0.275%

Kept pages

CrawlerPageTopicTokensWhen
brambleExploring Solana Applications | Solanamarkets q494820h ago
atlasThe Canton Development Stack - Canton Network Docsinfrastructure q478120h ago
duneOverview - Polygon Developer Docsmarkets q349920h ago
atlasMental Models for Canton - Canton Network Docsinfrastructure q477720h ago
brambleFeature: Accounts Index on Disk (1.14.17) - Releases - Solana Developer Forumsinfrastructure q320320h ago
brambleFeature: Compact Vote State (1.14.17) - Releases - Solana Developer Forumsinfrastructure q469920h ago
atlasModule 1: Understanding Canton - Canton Network Docsinfrastructure q348920h ago
duneIntroduction | Network | Nexus Docsmarkets q319820h ago
bramble02-25-23 Solana Mainnet Beta Outage Report | Solana Mediainfrastructure q52,21520h ago
cinderEJECTION_BALANCE review: improving UX, risk, and decentralisation · Issue #2883 · ethereum/consensus-specsinfrastructure q327220h ago
duneMargin Math | Math Engine | Nexus Docsdefi q54,84520h ago
atlasCanton's Solution: Three Pillars - Canton Network Docsinfrastructure q490320h ago
atlasThe Problem Canton Solves - Canton Network Docsinfrastructure q474920h ago
cinderEIP-7251: Raising Maximum Effective Balance For Validatorsinfrastructure q519,41020h ago
duneFunding Rate | Math Engine | Nexus Docsinfrastructure q55,71020h ago
atlasCanton Network in 5 Minutes - Canton Network Docsinfrastructure q354620h ago
duneClosed Loop | Math Engine | Nexus Docsinfrastructure q57,20820h ago
brambleA Framework for Governance - Introduction - #3 by cfl0ws - Governance - Solana Developer Forumsgovernance q318420h ago
duneSystem | Math Engine | Nexus Docsdefi q54,25520h ago
atlasUSDC Rewards - Coinbase Developer Documentationmarkets q320120h ago
atlasOverview - Coinbase Developer Documentationmarkets q336120h ago
duneGlobal Model | Math Engine | Nexus Docsdefi q521,10020h ago
cinderInformational | Ethereum Improvement Proposalsethereum q385720h ago
duneThe Engine, Formally | Math Engine | Nexus Docsmarkets q31,60720h ago
cinderEIP-1: EIP Purpose and Guidelinesethereum q57,44220h ago
duneBuilding the World’s Largest ZK Rollup with Mantlelayer 2 q496020h ago
cinder0x02 CSM Landscape - Proposals - Lido Governanceinfrastructure q52,97420h ago
cinderPolar - Delegate Thread - #23 by polar - Delegate Platform - Lido Governancegovernance q361720h ago
duneEVM Overview - Dune Docsethereum q315720h ago
bramblesolana.org/docs/SFDP_agreement_non-US.pdfsolana q55,23420h ago
brambleInstitutional Growth Lead- Japan @ Solana Foundationsolana q52,84220h ago
duneCurated Data Overview - Dune Docsethereum q331120h ago
brambleTokens | Solana Beachsolana q324020h ago
brambleAccount details | Solana Beachmarkets q54,09720h ago
cinderEthereum Node Operatorsmarkets q558,90620h ago
duneData Freshness - Dune Docsethereum q461820h ago
atlasOrder Lifecycle - Polymarket Documentationmarkets q356420h ago
brambleTokens | Solana Beachsolana q324020h ago
cinderSSV Lido Module (SSVLM) Proposal - #11 by Izzy - Proposals - Lido Governanceinfrastructure q324620h ago
duneData Trust & Reliability - Dune Docsinfrastructure q483720h ago

Newer page 21 of 48 Older