Plots of the Empirical Attainment Function

Computation and visualization of the empirical attainment function (EAF) for the analysis of random sets in multi-criterion optimization. M. López-Ibáñez, L. Paquete, and T. Stützle (2010) .

   differences between two bi-objective optimizers EAF differences between two variants of W-RoTS (color)

CRAN Status CRAN Downloads

[ Homepage ] [ GitHub ]

Maintainers: Manuel López-Ibáñez

Contributors: Manuel López-Ibáñez, Marco Chiarandini, Carlos M. Fonseca, Luís Paquete, and Thomas Stützle


The empirical attainment function (EAF) describes the probabilistic distribution of the outcomes obtained by a stochastic algorithm in the objective space. This R package implements plots of summary attainment surfaces and differences between the first-order EAFs. These plots may be used for exploring the performance of stochastic local search algorithms for biobjective optimization problems and help in identifying certain algorithmic behaviors in a graphical way.

The corresponding book chapter explains the use of these visualization tools and illustrate them with examples arising from practice.

Keywords: empirical attainment function, summary attainment surfaces, EAF differences, multi-objective optimization, bi-objective optimization, performance measures, performance assessment, graphical analysis, visualization.

Relevant literature:

  1. Manuel López-Ibáñez, Luís Paquete, and Thomas Stützle. Exploratory Analysis of Stochastic Local Search Algorithms in Biobjective Optimization. In T. Bartz-Beielstein, M. Chiarandini, L. Paquete, and M. Preuss, editors, Experimental Methods for the Analysis of Optimization Algorithms, pages 209–222. Springer, Berlin, Germany, 2010.
    (This chapter is also available in a slightly extended form as Technical Report TR/IRIDIA/2009-015). [ bibtex | doi:10.1007/978-3-642-02538-9_9 | Presentation ]

Download and installation

The software is implemented as an R package, but the code for only computing the EAF is available as a C program, and it does not require installing R or any R packages. Just download the package source code, uncompress it, and look for the directory inst/scripts/eaf. This code can be used to implement your own visualizations instead of the visualizations provided by the eaf package.

The visualization of the EAFs require installing the eaf package. Therefore, for making use of all the features, a basic knowledge of R is recommended. However, the eaf package contains two Perl scripts that allow to generate standard plots without any knowledge about R. See inst/scripts/eafplot/ and inst/scripts/eafdiff/ in the package source code. The scripts use the eaf package internally to generate the plots, and, hence, the eaf package must be installed and working.

The first step before installing the eaf package is to install R. Once R is installed in the system, there are two methods for installing the eaf package:

  1. Install within R (automatic download, internet connection required). Invoke R, then
        install.packages(c("modeltools", "eaf"))
  1. Download the package from CRAN (you may also need to download and install first the package modeltools), and invoke at the command-line:
        R CMD INSTALL <package>

where <package> is one of the three versions available: .tar.gz (Unix/BSD/GNU/Linux), .tgz (MacOS X), or .zip (Windows).

Search the R documentation if you need more help to install an R package on your system.

If you wish to be notified of bugfixes and new versions, please subscribe to the low-volume emo-list, where announcements will be made.

[ Download eaf package from CRAN ] [ Download Reference manual ] [ Development version (GitHub) ]


Once the eaf package is installed, the following R commands will give more information:

    example(eafdiffplot) # This one takes some time

Apart from the main R package, the source code contains the following extras in the directory inst/ (after installation, these files can be found at the directory printed by the R command system.file(package="eaf")):

  • scripts/eaf : This C program computes the empirical attainment function in 2 dimensions. It is NOT required by the other programs, but it is provided as a useful command-line utility. This version is based on the original code written by Carlos M. Fonseca available at Another version of the code that handles three dimensions is available at Prof. Fonseca's website.
  • scripts/eafplot : Perl script to plot summary attainment surfaces.
  • scripts/eafdiff : Perl script to plot the differences between the EAFs of two input sets.
  • extdata/ : Examples of utilization of the above programs. These are discussed in the corresponding chapter [1].

For more information, consult the README files at each subdirectory.


This software is Copyright (C) 2011 Carlos M. Fonseca, Luís Paquete, Thomas Stützle, Manuel López-Ibáñez and Marco Chiarandini.

This program is free software (software libre); you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

IMPORTANT NOTE: Please be aware that the fact that this program is released as Free Software does not excuse you from scientific propriety, which obligates you to give appropriate credit! If you write a scientific paper describing research that made substantive use of this program, it is your obligation as a scientist to (a) mention the fashion in which this software was used in the Methods section; (b) mention the algorithm in the References section. The appropriate citation is:

Manuel López-Ibáñez, Luís Paquete, and Thomas Stützle. Exploratory Analysis of Stochastic Local Search Algorithms in Biobjective Optimization. In T. Bartz-Beielstein, M. Chiarandini, L. Paquete, and M. Preuss, editors, Experimental Methods for the Analysis of Optimization Algorithms, pages 209–222. Springer, Berlin, Germany, 2010. doi: 10.1007/978-3-642-02538-9_9

Moreover, as a personal note, I would appreciate it if you would email [email protected] with citations of papers referencing this work so I can mention them to my funding agent and tenure committee.



  • Development version moved to GitHub:

  • Remove leading zeros from version number.

  • New parameters left.panel.last and right.panel.last of eafdiffplot.

  • Export and document function 'eafs' to compute EAFs.

  • Handle --colors=, --intervals=.

  • Fix crash in eafplot if 'sets' is a vector of strings.

  • Reset layout in eafdiffplot.

  • Compute eaf for 3D.

  • Added testthat testing framework.

  • The documentation is now generated with Roxygen2.

  • Entry points to C code are now properly registered.


  • Silence CRAN warning for GNU extensions in Makefiles in inst/scripts/eaf/Makefile.


  • Fix bug when automatically generating a legend in eafplot with the formula interface (Thanks to Bernd Bischl for reporting this)

  • Improve handling of various newline character formats.

  • Reduce memory consumption (up to four times less memory).

  • Mention option --legendpos=none to hide the legend.

  • Add options --maximise and --xmaximise and --colors=. Fix bug with --area.

  • eafplot.default now requires two colors when type=="area" and a palette is interpolated between these two colors for all the different levels plotted (Thanks to Alexandre Quemy for the suggestion).


  • Implement eafdiffplot(, type = "area"). This is now the default.

    The idea for the algorithm to compute the areas was provided by Carlos M. Fonseca. The implementation uses R polygons, which some PDF viewers may have trouble rendering correctly (See Plots will look correct when printed. To get the previous behaviour use eafdiffplot(, type = "point").

  • Attempt to deal with DOS/Unix newlines correctly.

  • Lots of cleanup. (--single, --output, --legend): New options. Default to PDF.

  • Many cleanups. EAF diff with type="area" is the default now. Parameter --cex replaced by --scale. Do not require ps2eps. Handle --obj1= better. Crop pdf before converting to png. (--noattsurfs): New option. (--output-dir,--output): New options. (--eps): New option. PDF is the default now.

  • Force eafdiff plots to use a square plotting region.

  • normalizes paths, thus it works with files such as ~/file.

  • Function eafdiffplot() handles percentiles=NA and grand.lines parameter.

  • Fix bug with ranges and maximise=TRUE in eafdiffplot().

  • Fix points.steps to work correctly for all values of maximise.

  • Fix bug in eafdiffplot(..., maximise=c(TRUE,FALSE), full.eaf = TRUE)

  • Fix "log" parameter in eafplot and eafdiffplot.

  • Avoid that eafplot.formula modifies global options.

  • Add 'axes' parameter to eafplot.

  • To avoid confusion, the arguments 'xaxis.side' and 'yaxis.side' of eafplot can only take values c("below", "above") and c("left", "right"), respectively.

  • Add missing Makefile in inst/scripts/eaf/.


  • Fix issues with only one point in the EAF.


  • Handle maximise argument in eafdiffplot and --maximise command-line option in

  • R/calls.R ( Fix problem with main parameter.


  • Initial release available in CRAN.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.


2.2 by Manuel López-Ibáñez, 2 months ago,

Report a bug at

Browse source code at

Authors: Manuel López-Ibáñez [aut, cre] , Marco Chiarandini [aut] , Carlos Fonseca [aut] , Luís Paquete [aut] , Thomas Stützle [aut] , Mickaël Binois [ctb]

Documentation:   PDF Manual  

GPL (>= 2) license

Imports modeltools, graphics, grDevices, stats, Rdpack

Suggests testthat, withr

System requirements: GNU make, Gnu Scientific Library

Suggested by ParamHelpers.

See at CRAN