2.4.2 Parameter Files: mdSim and mcSim (Prev) 2.5 Command Files (Next)

Example (ddSim):

Below is an example parameter file for a simple NVE ddSim parallel MD simulation for simulating a relatively small system (up 20000 atoms) on 8 processors in a 2 x 2 x 2 processor grid.

Simulation{
  Domain{
    gridDimensions    2    2    2
  }
  FileMaster{
    commandFileName   commands
    inputPrefix       in/
    outputPrefix      out/
  }
  nAtomType           1
  nBondType           1
  atomTypes           A    1.0
  AtomStorage{
    atomCapacity      12000
    ghostCapacity     12000
    totalAtomCapacity 20000
  }
  BondStorage{
    capacity          10000
    totalCapacity     20000
  }
  Buffer{
    atomCapacity       2000
    ghostCapacity      2000
  }
  pairStyle           LJPair
  bondStyle           HarmonicBond
  maskedPairPolicy    maskBonded
  reverseUpdateFlag   0
  PairPotential{
    epsilon           1.0
    sigma             1.0
    cutoff            1.122462048
    skin              0.4
    pairCapacity      40000
    maxBoundary       orthorhombic   30.0   30.0   30.0
  }
  BondPotential{
    kappa             400.0
    length            1.000
  }
  EnergyEnsemble{
    type              adiabatic
  }
  BoundaryEnsemble{
    type              rigid
  }
  NveIntegrator{
    dt                0.005
    saveInterval      0
  }
  Random{
    seed              8012457890
  }
  AnalyzerManager{
     baseInterval     1000
     OutputEnergy{
        interval        1000
        outputFileName  energy
     }
     OutputPressure{
        interval        1000
        outputFileName  pressure
     }
  }
}

In outline form, the format is

Simulation{
  Domain{ ... }
  FileMaster{ ... }
  nAtomType           1
  nBondType           1
  atomTypes           [string(name)]  [float(masss)]
                      ...
  AtomStorage{ ... }
  BondStorage{ ... }
  Buffer{ ... }
  pairStyle           [string]
  bondStyle           [string]
  maskedPairPolicy    [string]
  reverseUpdateFlag   [bool (0 or 1)]
  PairPotential{ ... }
  BondPotential{ ... }
  EnergyEnsemble{ ... }
  BoundaryEnsemble{ ... }
  NveIntegrator{ ... }
  Random{ ... }
  AnalyzerManager{ ... }
}

The main Simulation{} block in a ddSim simulation is associated with a DdMd::Simulation object. In a ddSim simulation, no distinction is made between a "Simulation" and a "System", and so there is no analog of the MdSystem or McSystem sub-blocks that appear in the mdSim and mcSim param files.

Many parameters and parameter blocks in this format are similar to those in mcSim and mdSim parameter files. The variables nAtomType, atomTypes, maskedPairPolicy, pairStyle, nBondType, bondStyle all have meaning as the corresponding parameters in mcSim and mdSim parameter files. So do corresponding variables associated with angles (nAngleType and nAngleStyle) and dihedral (nDihedralGroup and dihedralStyle) that do not appear in this example. The FileMaster, Random, EnergyEnsemble, and BoundaryEnsemble blocks are all identical to corresponding blocks in mcSim and mdSim parameter files.

The PairPotential and BondPotential blocks in this example are associated with instances of DdMd::PairPotential and DdMd::BondPotential, respectively These blocks take the same parameters as the MdPairPotential and BondPotential blocks of an mdSim simulation. The same pair and bond style strings are valid here as an mdSim or mcSim simulation. If the ddSim executable has been compiled with angle, dihedral, and/or external potentials enabled, and if one or more of these potentials has been enabled at run time by specifying nonzero values for nAngleType, nDihedralType or hasExternalPotential, then the PairPotential and BondPotential blocks must be followed by AnglePotential, DihedralPotential, and/or ExternalPotential blocks, as appropriate.

The AnalyzerManager block is associated with an instance of DdMd::AnalyzerManager, and has a format similar to that of the corresponding block in a mdSim or mcSim parameter file. This block must contain a value for the baseInterval, followed by zero or more polymorphic blocks, each of which contains the parameter block for a subclass of DdMd::Analyzer. The number of analyzers that are provided for use on-the-fly during ddSim simulations is thus far much smaller than the number avaiable for mdSim and mcSim simulations. This is partly a result of lack of time, and partly because some analyzers that are easy to implement in single-processor simulations are more difficult to implement efficiently in a parallel simulation.

reverseUpdateFlag

The reverseUpdateFlag is a bool variable whose value determines which of two communication patterns should be used in algorithm used to communicate particle data between neighboring processors. It should usually be set to zero. A value of 1 enables an algorithm in which the forces for each nonbonded or bonded group of particles in which particles are owned by different processors is calculated on only processor. This requires the resulting forces to then be communicate to the other processors via a separate "reverseUpdate" communication step. A value of 0 (the default) enables and algorithm in which this calculation is replicated on every processor that owns an atom within a group, which avoids the need to communicate forces arising from such group in a separate communication step. The reserveUpdate algorithm will be necessary for some integrators, but is generally slightly slower.

Domain

The Domain block is associated with a DdMd::Domain object. This object defines a processor grid, and controls the pattern of communication between neighboring processors within the grid. In the domain decomposition algorithm used by ddSim, the periodic simulation cell is divided into a regular grid of spatial domains, each of which is assigned to a different processor. The gridDimensions parameter is a vector of 3 integers (a Util::IntVector) that defines the dimensions of this grid (the number of processors) along each of the three spatial directions. The product of these three integers gives the total number of processors, which must agree with the number of processors that is requested from the operating system in the command line that runs the executable.

AtomStorage and BondStorage

The AtomStorage and BondStorage blocks are associated with DdMd::AtomStorage and DdMd::BondStorage objects. An AtomStorage is a container that holds DdMd::Atom objects for one processor. A BondStorage is a container that instead holds objects that represent covalent bonds, each of which contains references to two atoms. The parameter file for a ddSim simulation with angle and dihedral potentials enabled would also have AngleStorage and DihedralStorage blocks associated with containers for 3-body and 4-body covalent groups.

An AtomStorage maintains separate lists of two types of atoms. Local atoms are atoms that are "owned" by a processor. "Ghost" atoms are atoms that are "owned" by neighboring processors, but that are close enough to the boundaries between processor domains to interact with atoms owned by this processor.

The storage classes allocate blocks of memory at the beginning of a simulation, using "capacity" parameters that are specified by the user. The atomCapacity and ghostCapacity parameters gives the maximum number of local atoms and ghost atoms that can be accomodated on a single processor. The totalAtomCapacity is the maximum value for the total number of atoms on all processors. In the BondStorage block, capacity parameter gives the maximum total number of bonds on one processor, and totalCapacity gives the total number of bonds on all processors. Bonds (and other covalent groups, if used) are maintained in a single list that contains every bond that involves at least one atom that is owned by the processor.

The atomCapacity, ghostCapacity, and bondCapacity parameters must be chosen by the user to be large enough to accomodate any fluctuations in the number of atoms per processor. In a dense liquid containing a few thousand particles per processor, it is usually more than sufficient to set these capacities to be twice expected the average values, but a bit of experimentation is sometimes helpful. The totalAtomCapacity and totalBondCapacity must be greater tha or equal to than the total number of atoms or bonds, respectively, in the associated input configuration file. Using input values roughly twice these maximum values normally provides sufficient safety.

The log output produced by a ddSim simulation lists the actual maximum number of local atom and ghost atoms encountered on any processor during a simulation. Before running large simulations of a particular system, it is useful to run some short simulations and use these reported maximum values as a guide to the choice of appropriate (larger) capacity parameters.

Buffer

The Buffer block is associated with a DdMd::Buffer object that provides memory that is used for interprocessor communication. The atomCapacity parameters specify the maximum number of local atoms that may be communicated at once between neighboring processors when ownership is exchanged. Atom ownership is exchanged only when the neighbor list is rebuilt, which typically occurs about everten time steps. The ghostCapacity specifies the maximum number of ghost atom positions to be communicated at once between neighboring processors after every time step. These two parameters are used to decide how large a block of memory to allocate for use in interprocessor communication.

When setting up a first simulation for a particular system, one can be conservative and set these values larger than is expected to be necessary, e.g., by setting both parameters greater than or equal to the average total number of atoms per processor. If needed, values output in the log file for the maximum values of these parameters actually encountered during an initial simulation can then be used to adjust parameter values used in subsequent runs.

MD Integrator

The NveIntegrator block in the above example is a polymorphic block that must contain the file format for a subclass of DdMd::Integrator. Each subclass of DdMd::Integrator implements a parallel MD integrator. The DdMd::NveIntegrator class used in the above example implements a simple NVE velocity-Verlet algorithm. The most important parameter required for this integrator is a value for the time step size dt. The required parameters are different for different integrators, but always include a value for dt.

The Integrator block also contains a parameter saveInterval, which controls the frequency with which a restart (or checkpoint) file is rewritten. Setting saveInterval = 0, as in the above example, suppresses writing of the checkpoint file. The restart system is discussed in more detail in Sec. 2.10 Restarting.

2.4.2 Parameter Files: mdSim and mcSim (Prev) 2.4 Parameter Files (Up) 2.5 Command Files (Next)