Benchmarking results
This page presents, in downloadable form,
the database of measurements collected in the eBATS competition.
Measurements are collected by the BATMAN program;
measurements are displayed in a user-friendly format by the CAVE program.
Downloading the database
Here's how to download the current database:
wget http://ebats.cr.yp.to/results/20070214-atlas.bz2
wget http://ebats.cr.yp.to/results/20070214-cruncher.bz2
wget http://ebats.cr.yp.to/results/20070214-fireball.bz2
wget http://ebats.cr.yp.to/results/20070214-gggg.bz2
wget http://ebats.cr.yp.to/results/20070214-hald.bz2
wget http://ebats.cr.yp.to/results/20070214-katana.bz2
wget http://ebats.cr.yp.to/results/20070214-mace.bz2
wget http://ebats.cr.yp.to/results/20070214-neumann.bz2
wget http://ebats.cr.yp.to/results/20070214-pclin118.bz2
wget http://ebats.cr.yp.to/results/20070214-pclin153.bz2
wget http://ebats.cr.yp.to/results/20070214-poema.bz2
wget http://ebats.cr.yp.to/results/20070214-shell.bz2
wget http://ebats.cr.yp.to/results/20070214-td152.bz2
wget http://ebats.cr.yp.to/results/20070214-td156.bz2
wget http://ebats.cr.yp.to/results/20070214-td158.bz2
wget http://ebats.cr.yp.to/results/20070214-td159.bz2
wget http://ebats.cr.yp.to/results/20070214-td162.bz2
wget http://ebats.cr.yp.to/results/20070214-td178.bz2
wget http://ebats.cr.yp.to/results/20070214-td185.bz2
wget http://ebats.cr.yp.to/results/20070214-td186.bz2
wget http://ebats.cr.yp.to/results/20070214-td189.bz2
wget http://ebats.cr.yp.to/results/20070214-thoth.bz2
wget http://ebats.cr.yp.to/results/20070214-whisper.bz2
New database entries are added whenever more benchmarks are completed.
Database format
Each database entry is a line consisting of the following space-separated words:
- BATMAN version; e.g., 20070214.
- Computer name; e.g., katana.
There is a
separate page providing more information about the computers:
e.g., katana's CPU is a 2137MHz Intel Core 2 Duo (6f6).
- Benchmark start date; e.g., 20070215.
- BAT name; e.g., ronald.
- BAT version; e.g., 3.
- BAT parameter list; e.g., 4096.
- Operation or object measured; e.g., signedmessage.
- Number of bytes in the original (unencrypted, unsigned) message
(or - for operations not involving a message); e.g., 1109.
- Type of measurement; e.g., cycles.
- Measurement, or median of all measurements if there were multiple measurements:
e.g., 79319912.
The ronald-3 BAT with parameter 4096 (4096-bit RSA key)
used 79319912 cycles on katana to sign an 1109-byte message.
- All measurements if there were multiple measurements:
e.g.,
79426336 79064112 79089856 79521104 79111288
79307056 80360128 79402624 79512728 79319912
79312904 79251560 79416288 79289520 80807048.
The lines are collected into bzip2-compressed files
indexed by BATMAN version and computer name.
There are four types of measurements in the database.
First, there are operations whose speed is measured by BATMAN, measurement type cycles:
- ciphertext, for encrypting BATs:
encrypting a message.
- keypair, for all BATs:
generating a secret key and public key.
- messagesigned, for signing BATs:
verifying a signed message and a public key,
returning the original message that was signed.
- plaintext, for encrypting BATs:
decrypting an encrypted message.
- sharedsecret, for secret-sharing BATs:
computing a shared secret.
- signedmessage, for signing BATs:
signing a message.
Second, there are objects whose size is measured by BATMAN, measurement type bytes:
- ciphertext, for encrypting BATs:
an encrypted message.
- messagesigned, for signing BATs:
a message extracted from a signed message.
Should always be the same as the original message.
- plaintext, for encrypting BATs:
a message produced by decrypting an encrypted message.
Should always be the same as the original message.
- publickey, for all BATs:
a public key.
- secretkey, for all BATs:
a secret key.
- sharedsecret, for secret-sharing BATs:
a shared secret.
- signedmessage, for signing BATs:
a signed message.
Third, there are objects reported by the BAT:
- ccattacks, measurement type attack, for encrypting BATs:
100 if adaptive chosen-ciphertext attacks (IND-CCA2) are more effective than chosen-plaintext attacks;
or 0 if adaptive chosen-ciphertext attacks are no more effective than chosen-plaintext attacks.
-1 means unreported.
- cdhchance, measurement type chance, for secret-sharing BATs:
the log base 2 of the number of euros spent by the attacker;
then the log base 2 of the number of keys simultaneously attacked;
then 256 times the log base 0.5 of the probability
that an attacker spending 1 second will succeed at deducing at least one shared secret;
then the same for an attacker spending 2 seconds;
then the same for an attacker spending 4 seconds;
and so on through 2^40 seconds.
-1 means unreported.
- copyrightclaims, measurement type restriction, for all BATs:
0 if there are no known present or future claims by a copyright holder
that the distribution of this software infringes the copyright;
10 if the author is aware of third parties making such claims,
but the authors disputes those claims;
20 if the author is aware of third parties making such claims,
and the author agrees with the claims,
but the author has no financial connections to the copyright;
30 if the author has financial connections to a copyright restricting
distribution of the software.
-1 means unreported.
- distinguishingchance, measurement type chance, for encrypting BATs:
the log base 2 of the number of euros spent by the attacker;
then the log base 2 of the number of keys simultaneously attacked;
then 256 times the log base 0.5 of the ciphertext-distinguishing (IND-CPA) probability
for an attacker spending 1 second;
then the same for an attacker spending 2 seconds;
then the same for an attacker spending 4 seconds;
and so on through 2^40 seconds.
-1 means unreported.
- fakekeyattacks, measurement type attack, for secret-sharing BATs:
100 if an active attacker can save time by providing fake keys
(in applications that do not go to any extra effort to validate keys);
or 0 if an active attacker obtains no benefit from fake keys
(for example, if the sharedsecret function includes all necessary key validation).
-1 means unreported.
- forgerychance, measurement type chance, for signing BATs:
the log base 2 of the number of euros spent by the attacker;
then the log base 2 of the number of keys simultaneously attacked;
then 256 times the log base 0.5 of the probability
that an attacker spending 1 second will succeed at forging at least one signed message;
then the same for an attacker spending 2 seconds;
then the same for an attacker spending 4 seconds;
and so on through 2^40 seconds.
-1 means unreported.
- patentclaims, measurement type restriction, for all BATs:
0 if there are no known present or future claims by a patent holder
that the use of this software infringes the patent;
10 if the author is aware of third parties making such claims,
but the authors disputes those claims;
20 if the author is aware of third parties making such claims,
and the author agrees with the claims,
but the author has no financial connections to the patent;
30 if the author has financial connections to a patent restricting
use of the software.
-1 means unreported.
- timingattacks, measurement type attack, for all BATs:
0 if the software does not leak
any secret information through timing
(variable time for branching, variable time for memory access, etc.);
or 100 if the software leaks secret information through timing.
-1 means unreported.
Fourth, there are some extra CPU notes and tuning notes included in the database,
in unspecified formats subject to change:
- bat_tuning:
the BAT tuning, or - for untuned BATs.
- compiler:
the compiler used, including options;
e.g., gcc_-m64_-march=k8_-O2_-fomit-frame-pointer.
- compiler_version:
e.g., 4.0.3_(Ubuntu_4.0.3-1ubuntu5).
- cpucycles_implementation:
e.g., amd64cpuinfo.
- cpucycles_persecond:
the number of cycles carried out by the CPU in one second.
- cpuid:
details of the CPU being used;
e.g., GenuineIntel-000006f6-bfebfbff_.
Version
This is version 2007.06.24 of the eBATS results.html web page.
This web page is in the public domain.