our mascot GIZMO Users Guide

Table of Contents

  1. Code Overview
  2. Feature Set (Types of Physics Modules)
  3. Code Use, Authorship, Citation, Sharing, & Development Rules
  4. Fluid (Hydro) Solvers
  5. Similarities & Differences from GADGET (+other codes)
  6. Compiling and Using the Code (a very brief tutorial)
  7. Modules in Config.sh (Setting compile-time options, i.e. Physics)
  8. The Parameterfile (Setting run-time parameters)
  9. Snapshot & Initial Condition Files
  10. Log Outputs & Diagnostic Files
  11. Test Problems
  12. Useful Additional Resources
  13. Frequently Asked Questions
  14. Disclaimer

1. Code Overview

Welcome to GIZMO.

The simulation code GIZMO is a flexible, massively-parallel, multi-method multi-physics code, with a wide range of different physics modules described in the “Feature Set” below. The code is descended from P-GADGET, itself descended from GADGET–2, and is fully compatible with GADGET analysis codes, initial conditions, and snapshots. GIZMO is not an acronym – it refers both to the code’s multi-purpose applications and to its historical relationship to GADGET.

The code is extensively documented. Please read the User Guide before asking questions of me or other users: most questions I get are already answered here.

The code is written in C++ (C++17 standard), and should run on all parallel platforms that support MPI. The portability of the code has been confirmed on a large number of systems ranging from a laptop to >1 million threads/cores on national super-computers, including GPU-accelerated systems via the Kokkos portability layer.

The original reference for the numerical methods (the original fluid+gravity solver) is the paper here. It is recommended that you read it before using (or modifying) the code, although of course there have been many updates and improvements (including in the core algorithms) since this paper, for which this User Guide provides many more details. For more details on GADGET, which underpins some aspects here, see the GADGET–2 methods paper or GADGET–2 user guide.

The un-official code website (showing examples, demonstrations, and giving an overview of the code) is here. The code is fully public, and is hosted on GitHub here: that repository holds the stable branch (the default branch) and the development branches, and there is no separate private code. The older repositories (the old public git, and the public and private bitbucket repositories) persist only for archival reasons: they were last synced in October 2026 and are no longer maintained.

GIZMO was written by me, Philip F. Hopkins (PFH), although the codebase builds on parts of Volker Springel’s GADGET code. If you find the code useful, please reference the numerical methods paper in all studies using simulations run with GIZMO. You should also reference the GADGET paper for the domain decomposition and N-body algorithms. Various modules have their own methods papers, contributed by different authors, that should be cited if they are used; these are specified in the “Config.sh (Setting compile-time options)” section of this Guide.

Code Philosophy & Design

A few core principles distinguish GIZMO from other simulation codes and are worth understanding before diving into the technical details.

Modularity and cross-compatibility. Almost every physics module in the code is designed to be independently toggled on or off via a single flag in Config.sh, and modules are designed to be cross-compatible wherever physically meaningful. A simple periodic-box hydrodynamics simulation can be transformed into a simulation of star formation in a magnetized giant molecular cloud, live thermonuclear burning in a white dwarf merger, elastic solid-body collisions and giant planetary impacts, dust particulate dynamics with aerodynamic drag and Lorentz forces, cosmic-ray magneto-hydrodynamics with anisotropic transport and streaming, radiation-driven ionization fronts and photo-dissociation regions, or any combination of the above – with just a few compile-time switches in Config.sh, often without modifying a single line of source code. This modularity means that improvements to any one module (e.g. a more accurate cooling network, or a GPU-accelerated neighbor-list hydro solver) automatically benefit every science application that uses it, and entirely new physics can be grafted in without risk of breaking existing functionality. The code is designed with the explicit philosophy that physical and numerical flexibility should be a first-class feature, not an afterthought: the same codebase that runs a pure dark matter cosmological simulation should, with a handful of flags, become a multi-physics star and planet formation code or a nuclear astrophysics simulation engine.

Multi-method flexibility. Unlike most astrophysical simulation codes, which are built around a single numerical approach (e.g. AMR, or SPH, or a moving mesh), GIZMO implements fundamentally different fluid solvers – meshless finite-mass (MFM), meshless finite-volume (MFV), smoothed-particle hydrodynamics (SPH), and Eulerian fixed-grid methods – within the same codebase, sharing the same gravity solver, the same physics modules, and the same I/O infrastructure. This allows direct, controlled comparisons of numerical methods on the same problem, and means users can choose the most appropriate method for their science rather than being locked into a particular approach.

Accuracy and conservation. The default MFM/MFV methods are designed to combine the best properties of particle-based and grid-based methods: exact mass, energy, and momentum conservation (to machine precision), Galilean invariance, zero advection errors, automatic and continuous spatial adaptivity, and high-order shock capturing via Riemann solvers. These properties are not compromises – they are achieved simultaneously, in a mathematically well-defined framework. See Hopkins 2015 for the mathematical foundations.

Scalability. The code is designed for problems ranging from a few hundred particles on a laptop to billions of resolution elements on national supercomputing facilities. MPI + Kokkos performance-portable execution (CUDA, HIP, or OpenMP backends) plus hierarchical adaptive timestepping enable efficient use of modern heterogeneous computing architectures. The compute-intensive physics — hydro chain, gravity tree, cooling/chemistry, nuclear networks, RT, feedback, sinks — runs entirely on the device (GPU or OpenMP threads); the host MPI process is primarily a memory manager and inter-rank dispatcher. For details, see the Performance-Portable Heterogeneous Execution section below.

2. Feature Set

The GIZMO code is a flexible, massively parallel, multi-purpose fluid dynamics + gravity code. Some (not all!) current physics include:

Examples of Different Simulations Run with GIZMO

GIZMO and its methods have been used in a large number of simulations and well over a hundred different papers (for a partial list of works which reference the original methods paper, click here). Some examples are shown above, including: (a) Cosmology: Large-volume dark matter+baryonic cosmological simulation (Davé et al. 2016). (b) Galaxy formation: with cooling, star formation, and stellar feedback (Wetzel et al. 2016). (c) Dust (aero)-dynamics: grains moving with Epstein drag and Lorentz forces in a super-sonically turbulent cloud (Lee et al. 2017). (d) Black holes: AGN accretion according to resolved gravitational capture, with feedback driving outflows from the inner disk (Hopkins et al. 2016). (e) Proto-stellar disks: magnetic jet formation in a symmetric disk around a sink particle, in non-ideal MHD (Raives 2016). (f) Elastic/solid-body dynamics: collision and compression of two rubber rings with elastic stresses and von Mises stress yields. (g) Plasma physics: the magneto-thermal instability in stratified plasma in the kinetic MHD limit (Hopkins 2017). (h) Magneto-hydrodynamics: the Orszag-Tang vortex as a test of strongly-magnetized turbulence (Hopkins & Raives 2016). (i) Star formation: star cluster formation (with individual stars) including stellar evolution, mass loss, radiation, and collisionless dynamics (Grudic et al. 2016). (j) Fluid dynamics: the non-linear Kelvin-Helmholtz instability (Hopkins 2015). (k) Multi-phase fluids in the ISM/CGM/IGM: ablation of a cometary cloud in a hot blastwave with anisotropic conduction and viscosity (Su et al. 2017). (l) Impact and multi-material simulations: giant impact (lunar formation) simulation, after impact, showing mixing of different materials using a Tillotson equation-of-state (Deng et al. 2017).

Code Scaling (Parallelism & Performance)

GIZMO uses a hybrid MPI+OpenMP parallelization strategy with a flexible domain decomposition and hierarchical adaptive timesteps, which enable it to scale efficiently on massively-parallel systems with problem sizes up to and beyond billions of resolution elements. In addition, we have implemented a large number of optimizations for different problems and physics, including e.g. rolling sub-routines and communication together, implementing problem-specific weighting schemes for domain decomposition, solving gravity with a hybrid tree-nested particle mesh approach, and adopting fully adaptive spatial and time resolution. Remarkably, despite the additional computational complexity, the new fluid solvers are actually faster in GIZMO than their SPH counterparts, owing both to the reduced neighbor number needed and to several custom optimizations.

Code scalings are always (highly) problem-and-resolution-dependent, so it is impossible to make completely general statements about how GIZMO scales. However, we can illustrate one example here. This shows the code scalings of GIZMO (in hybrid MPI+OpenMP mode) on full production-quality FIRE–2 cosmological simulations of galaxy formation, including the full physics of self-gravity, hydrodynamics, cooling, star formation, and stellar feedback (from Hopkins et al. 2017, arXiv:1702.06148).

Weak scaling (making the computational problem larger in size, but increasing the processor number proportionally) for a full cosmological box, populated with high resolution particles (fixed particle mass), run for a short fraction of the age of the Universe. We increase the cosmological volume from 2 to 10,000 cubic comoving Mpc. In these units, theoretically “ideal” weak scaling would be flat (dashed line shown). The weak scaling of GIZMO’s gravity+MHD algorithm is near-ideal (actually slightly better at intermediate volume, owing to fixed overheads and statistical homogeneity of the volume at larger sizes) to greater than a million threads (>250,000 CPUs and >16,000 independent nodes).
Strong scaling (CPU time on a fixed-size problem, increasing the processor number) for a "zoom-in” simulation of a Milky Way-mass galaxy (1e12 solar-mass halo, 1e11 solar-masses of stars in the galaxy) or a dwarf galaxy (1e10 solar-mass halo, 1e6 solar-masses in stars in the galaxy), each using 1.5e8 (150 million) baryonic particles in the galaxy, run to 25% of the present age of the Universe. In these units, “ideal” scaling is flat (dashed line), i.e. the problem would take same total CPU time (but shorter wall-clock time as it was spread over more processors). Our optimizations allow us to maintain near-ideal strong scaling on this problem up to ~16,000 cores per billion particles (~2000 for the specific resolution shown).
Test increasing the resolution instead of the problem size (specifically, increasing the particle number for the same Milky Way-mass galaxy, while increasing the processor number proportionally). Because the resolution increases, the time-step decreases (smaller structures are resolved), so the ideal weak scaling increases following the dashed line. The achieved scaling is very close to ideal, up to >16,000 cores for ~1 billion particles.

This is an extremely inhomogeneous, high-dynamic-range (hence computationally challenging) problem with very dense structures (e.g. star clusters) taking very small timesteps, while other regions (e.g. the inter-galactic medium) take large timesteps — more “uniform” problems (e.g. a pure dark-matter only cosmological simulation or pure-hydrodynamic turbulence simulation) can achieve even better scalings.

Important Notes on Stable vs. Development Branches

The whole code is public: there is no separate private or proprietary version. The repository holds a stable branch (its default branch) and a number of development branches. Users are strongly encouraged to default to the stable branch. Development branches are intended for active development and testing: nothing in them should be taken as production code, or assumed to be ready for widespread use.

If you want to use a module that is still in development (in a development branch, or described in this Guide or in Template_Config.sh as in development), it is highly recommended that you first contact its developers directly – they are identified in this Guide, in the Template_Config.sh file, and in the code comments. Otherwise there is a high chance it will be broken or – worse – that it will compile and run but, outside of very specific module and parameter combinations, will simply not give correct answers nor behave as intended.

3. Code Use, Authorship, Citation, Sharing, & Development Rules

Use, Authorship, & Citation Requirements

Citation Requirements:

Any paper using GIZMO should, at a minimum, cite the methods paper: Hopkins 2015, MNRAS, 450, 53 (“A New Class of Accurate, Mesh-Free Hydrodynamics Methods,” arXiv:1409.7395). You should also cite the GADGET methods paper (Springel, 2005, MNRAS, 364, 1105) for the domain decomposition and N-body algorithms.

When using specific modules, the relevant code papers must also be cited: for example, papers using the MHD version of the code should cite the MHD methods paper: Hopkins & Raives 2016, MNRAS, 455, 51 (“Accurate, Meshless Methods for Magneto-Hydrodynamics,” arXiv:1505.02783). Anisotropic diffusion operators (conduction, viscosity, radiation, cosmic rays, non-ideal MHD, passive scalar diffusion) should cite Hopkins 2017, MNRAS, 466, 3387 (“Anisotropic Diffusion in Mesh-Free Numerical Magnetohydrodynamics,” arXiv:1602.07702). The citations required for the various code modules are all given in the “Config.sh (Setting Compile-Time Options)” section of this Guide, where the modules are described.

Remember, these modules were developed at great expense in both time and grant funding by their authors. Moreover readers may want to see more detailed numerical tests. So please, respect the citation requirements for all modules: appropriate citation, as requested in this Guide and for the different modules, is the expectation and requirement for any use of GIZMO in published scientific work.

Authorship Requirements & Acceptable Use:

There are no requirements for offering authorship of papers resulting from the code. The requirement for use of GIZMO in published scientific work is appropriate citation, as requested in this Guide and for the different modules (above). If you use a module that is still in development, please contact its developers first (see Notes on Stable vs. Development Branches).

Code Sharing Guidelines

The code is free software, distributed under the GNU General Public License. This implies that you may freely distribute and copy the code. You may also modify it as you wish, and distribute these modified versions as long as you indicate prominently any changes you made in the original code, and as long as you leave the copyright notices, and the no-warranty notice intact. Please read the General Public License for more details. Note that the authors retain their copyright on the code.

Code Development & Extensions (New Physics and Code)

You are encouraged to study and modify this code! Please, if you have ideas for interesting physics to add, add it! You are only asked to do a few things, to make your experience and that of everyone else much easier.

General Code-style Principles (Good Practices):

4. Fluid (Hydro) Solvers

GIZMO is unique in that it allows the user (at compile time) to choose the method with which they would like to solve the fluid (hydrodynamic, MHD, RHD) equations. This flexibility goes far deeper than in most codes. We’re not talking about the difference between e.g. different Riemann solvers or different ways of writing the SPH equation of motion, we’re talking about different methods for how the simulation volume is decomposed (e.g. the “grid” underlying everything).

Examples (Demonstrations, Animations, and Talks)

For a variety of images and movies demonstrating how these produce different results in real simulations (a few random examples are shown above), and highlighting the regimes where the new Lagrangian Godunov-type methods (MFV and MFM) in GIZMO have unique advantages, see the code overview site here.

Much more detailed quantitative discussions are found in the GIZMO methods paper here and the follow-up GIZMO MHD methods paper here, or see my notes here from a summer-school lecture for a broader overview of different methods (SPH, AMR, MFM/MFV, moving-Voronoi-meshes).

Mesh Structure: Overview of the Different Methods

Below are the current solvers in GIZMO, with some descriptions. For more detail, see the GIZMO methods paper. The illustration above demonstrates some of the key conceptual differences between some of the methods here. For an irregularly distributed set of sampling/grid points or “particles” (black circles) with locations \(x_{i}\), any simulation code requires a way to partition the volume to solve the equations of fluid dynamics between them. Left: The meshless finite-mass/finite-volume (MFM and MFV) methods here. The volume partition is given by a smooth kernel decomposition at every point (red/green/blue channels represent the fraction of the volume at each point associated with the corresponding particle). This resembles a Voronoi tessellation with smooth edges. Despite the kernel function being spherical, the domains associated with each particle are not, and the entire volume is represented. The fluid equations are then solved by integrating over the domain of each particle/cell. Center: The unstructured/moving Voronoi-mesh partition. Now the boundaries are strict step functions at the faces given by the tessellation. Note that this is (exactly) the limit of our MFM/MFV method for an infinitely sharply-peaked kernel function. The volume integrals are then reduced to surface integrals across the faces. Right: The SPH partition. In SPH the contribution to volume integrals behaves as the kernel, centered on each particle location; the whole volume is “counted” only when the kernel size is infinitely large compared to the inter-particle spacing (number of neighbors is infinite). The equations of motion are evaluated at the particle locations \(x_{i}\), using the weighted-average volumetric quantities from the volume partition.

If you are interested in additional comparisons, see the methods paper for a more technical description specific to GIZMO, or see my notes here from a summer-school lecture for a broader overview of different methods (SPH, AMR, MFM/MFV, Voronoi-meshes).

MFV (Meshless Finite-Volume) Method:

Enabled by HYDRO_MESHLESS_FINITE_VOLUME: This method was developed in pieces by many people in the fluid dynamics community, and the foundations for this implementation (as well as various convergence proofs, etc) are given in Lanson & Vila 2008 (papers here and here). It is an evolution of the first implementation for astrophysical use in Gaburov & Nitadori 2010 (paper here). The method uses a kernel function to partition the volume (unlike SPH, the partition is analytically exact and only serves to divide the volume – it is not a ‘smoothing function’); it then derives a second-order accurate and consistent set of hydrodynamic equations from that partition. The discretized equations define an “effective face” for the particles (the “partition generating points” the same way that particles define the “mesh generating points” for Voronoi-type mesh methods), across which an exactly conservative Riemann problem is solved with a Godunov-type scheme. The method is simultaneously second-order accurate and fully conservative (see the methods paper for demonstrations). It is very similar to moving-mesh methods using Voronoi or other geometric tesselation, and captures contact discontinuities more accurately than any of the other methods below. Particles will gain and lose mass according to the motion at the faces – the actual motion of the mesh-generating points is, in principle, arbitrary. In the default mode, the mesh-generating points move with the center-of-mass velocity of the cells (which still means there are mass fluxes, but they are relatively small); however as described in the section on mesh motion below, the mesh motion can in principle be arbitrarily specified in this method. If desired, on-the-fly (fully conservative) particle splitting and merging prevents particle masses from becoming too disparate.

MFM (Lagrangian Meshless Finite-Mass) Method:

Enabled by HYDRO_MESHLESS_FINITE_MASS: This method is similar to the meshless finite volume method, except that when solving the Riemann problem between particles, we implicitly assume a deformation of the face between the particles such that there is no mass flux between the particles (essentially, we solve the Riemann problem not for the “fixed” face located halfway between the two particles, but for the -Lagrangian- face moving with the mass sheet that defined the “original” face). Thus the particle masses stay fixed (at least under hydro operations). The elements are therefore necessarily moving with the mean fluid flow (one cannot, in MFM, force the mesh motion to be different from the fluid motion). The method is therefore “Lagrangian” in that fluid elements can be trivially tracked with time. As shown in the methods paper, this is still a second-order consistent method. However, it does introduce a second-order (non-linear) error in the advection equation: basically, when we implicitly re-map to the next timestep, the “new” face does not exactly match the Lagrangian deformation used in the step before. So we trade a small amount of accuracy for certain types of problems at fixed resolution for the convenience of tracking a fixed Lagrangian fluid element and guaranteeing that we don’t have wildly different particle masses interacting gravitationally (which can seed noise). On some problems where advection would be very large and the problem is coupled to self-gravity (so differing particle masses can introduce noise), this is the most accurate of the methods in GIZMO. This is the “default” method of the code if no method is specified.

Regular-Mesh (Cartesian Fixed-Grid) Method:

Enabled by HYDRO_REGULAR_GRID, this is a standard finite-volume Godunov scheme solved on a fixed (non-moving), regular Cartesian mesh. This is built on the architecture of the MFV method, so the mesh-generating points are simply fixed relative to one another (a uniform boost of the whole system is trivial to add, if desired, but would have to be hard-coded for the problem), using the assumption of a geometric grid to exactly calculate closed, regular geometric faces. This is an “Eulerian” or “fixed-grid” method. However, mesh motion is allowed, so long as it preserves the regularity of the mesh (see the discussion in the mesh motion section, below) – for example, the mesh can be uniformly shearing or boosted. This mode also can used to define grids of other regular geometries (e.g. cylindrical or spherical grids) – in those cases the mesh can be rotating or spherically expanding/contracting with regular speeds, as well. For some problems (e.g. highly sub-sonic problems, where the constant mesh deformation implicit in MFM or MFV can seed sound wave), this can be more accurate or give smoother solutions, but it encounters challenges when the fluid is moving rapidly over the mesh or collapsing under self-gravity.

SPH (Smoothed-Particle Hydrodynamics):

This is SPH, well-known from astrophysics and fluid dynamics (going back to Gingold & Monaghan 1977). A kernel function is used to smooth volumetric field quantities. From these approximate (smoothed values), the equations of motion are derived. All the formulations of SPH supported here are derived from the modern “Lagrangian” formulation (see Springel & Hernquist 2002), in which the equations of motion are derived from a particle Lagrangian, and so are manifestly conservative. All of the SPH methods used, by default, include the most current bleeding-edge formulations of things like artificial dissipation (viscosity, conductivity, diffusion), higher-order kernels, corrections for adaptive kernel softenings, timestepping, etc.

The SPH methods supported come in two flavors:

(a) Enabled by HYDRO_PRESSURE_SPH: This enables the “pressure SPH” or “density-independent SPH” from Hopkins 2013 (see paper here) or Saitoh and Makino 2013. This computes both the density and pressure (all volumetric quantities) from direct kernel-smoothing, and self-consistently propagates these into the equation of motion. This solves the “surface tension” problem well-known in “traditional” SPH (in which the two sides of a contact discontinuity see different pressures, leading to a surface-tension-like repulsive term across the discontinuity). The tradeoff for this is greater noise and diffusion at pressure discontinuities (shocks), as well as less accurate energy conservation in radiative cooling steps.

(b) Enabled by HYDRO_DENSITY_SPH: This is the “traditional” or “density” form of SPH, in which only the density is kernel-averaged, and the pressure is computed from this plus the particle-carried entropy or internal energy.

Note that –any– SPH method, no matter how accurate the artificial dissipation terms is, or which form of the equation of motion is used, is not second-order consistent. This is the origin of the well-known fact that SPH only converges (formally speaking) in the limit where the number of neighbors inside the kernel goes to infinity at the same time that the total number of neighbors also goes to infinity. See the methods paper for more discussion of this point. As a result, for almost any problem you can think of, SPH at fixed resolution is substantially less accurate, and converges less rapidly, compared to other methods in GIZMO. It is, however, exceptionally stable (numerically speaking): in conditions which would crash any other hydro solver, SPH will keep running (this is not necessarily an advantage, however: if it would crash the other solvers, SPH may well give you the –wrong– answer. It will just give you an answer.)

Moving Voronoi Meshes:

Moving Voronoi-type meshes (with exact geometric tessellation, as opposed to the MFM/MFV volume partition) are not currently available for general use. The MFV and MFM volume partitions provide similar functionality for most applications.

Mesh Motion:

Above, we described different methods (MFM, MFV, regular grids, SPH, Voronoi-meshes) which refer to the manner in which the volume is partitioned among mesh-generating points or fluid elements, or equivalently the geometry of the mesh (or the way effective “faces” are defined between elements for purposes of computing the fluxes).

However, this is (at least for some methods) separate from the question of how the mesh moves. Some methods above (e.g. MFM or SPH) require certain assumptions about the mesh motion, while others (e.g. MFV) allow essentially arbitrary mesh motion (“upon which” the fluid equations are solved each timestep with the method). GIZMO allows the user to define essentially any arbitrary mesh motion desired.

Some examples of different kinds of mesh motion allowed in GIZMO are shown in the Figure above. Broadly-speaking, there are three categories into which different mesh-motion schemes fall:

Eulerian (Fixed-Volume or Non-Moving) Meshes:

The simplest “Eulerian” schemes, and still the standard for traditional fixed-grid or AMR methods, do not allow the mesh to move at all. The mesh-generating points (which determine the locations of the “cells” or “particles”) are simply “frozen” and the fluid is advected over them.

This is the default behavior if you use the REGULAR_GRID fluid method (enabled with the HYDRO_REGULAR_GRID flag), since the grid usually is static in time.

You can also force it to be true with other methods, like MFV, by setting the compile-time parameter HYDRO_FIX_MESH_MOTION=0. The mesh can then have an arbitrary geometry, but not move. Note, though, that this can often lead to some nasty errors if your mesh is not well-ordered initially (e.g. if the cells are asymmetric, and the mesh cannot move to “regularize” then, you will imprint this asymmetry onto the global solution).

Although the discussion above in the section about different methods (and in the linked talks) describe many advantages of moving meshes or “Lagrangian” methods for problems with large bulk motion, advection, gravitational collapse, etc, there are certainly problems where fixed (non-moving) grids can still be advantageous (if the grid is well-ordered or regular). For example, on problems where you wish to follow highly sub-sonic motion accurately, moving meshes generate “grid noise” because the mesh is constantly moving and deforming. This deformation of the system generates pressure fluctuations on a scale of order the integration error, which in turn launch sound waves, that can significantly corrupt the desired behavior. A regular (e.g. Cartesian) non-moving mesh has no “grid noise” however, so avoids these issues. Of course, the method is more diffusive if the grid is not moving, especially if there is substantial advection across the cells, and one has to be careful about imprinting preferred directions on the flow. But in general, in problems where it is very important to maintain a “smooth” background flow (e.g. reduce noise at all costs, even at the cost of significant numerical diffusivity), this can be useful.

Lagrangian (Fixed-Mass) Meshes Moving with the Fluid:

At the opposite extreme, in traditional “Lagrangian” schemes, the mesh moves with the fluid velocities exactly. Specifically, the mesh-generating points move with the center-of-mass fluid velocity of the cell/element. This will of course lead to arbitrary deformations of the mesh, so your method needs to be capable of handling this (it is simply not possible to maintain a regular Cartesian “cuboid” mesh while advecting all the cells with arbitrary local fluid velocities, for example).

In MFM or SPH, the method solves the fluid equations for a fixed mass element. Therefore, the mesh must move with the mean fluid motion, or else the equations give contradictory and meaningless (unphysical) results. These methods follow fixed mass elements and the mesh is continuously moving and de-forming along with the mean fluid flow (determined self-consistently according to the hydrodynamic equations). They are therefore traditional “Lagrangian” methods, and the mesh will always behave this way if they are chosen as the fluid method.

Usually, with other quasi-Lagrangian methods (e.g. MFV), this is also the preferred (default) motion of the mesh, as described below. This is for example, the behavior of MFV used in the original GIZMO methods paper and in the work by Gaburov & Nitadori 2010. One can force MFV to behave this way by setting the compile-time parameter HYDRO_FIX_MESH_MOTION=7.

These schemes have the advantages discussed above, for problems involving e.g. highly super-sonic motion, gravitational collapse, angular momentum, etc, especially when the “preferred” geometry of the problem is unknown (or un-defineable), so one does not wish to imprint preferred directions on the code and wishes to minimize advection errors as much as possible.

Meshes with Arbitrarily User-Defined Motion (Arbitrary Lagrangian-Eulerian or Hybrid Methods):

In principle, however, some methods (such as the MFV method or moving Voronoi-mesh-type methods) allow for arbitrary mesh motion – the mesh can move with the fluid, or not. GIZMO allows the User, if desired, to control the mesh motion as they like, by setting the compile-time parameter HYDRO_FIX_MESH_MOTION.

This can be set to any of several values, which determine the mesh motion:

This allows a tremendous amount of flexibility for different problems. However note that you should be careful setting the mesh motion to “custom” values. As noted, MFM and SPH require the mesh move with the fluid, and will always enforce this. If you use the REGULAR_GRID setting, and choose any type of mesh motion (other than HYDRO_FIX_MESH_MOTION=0), you must be careful that the motion you choose actually preserves the regularity of the grid. A shearing box with a Cartesian, cuboid grid is a good example where this is well-defined: the different “rows” of the grid can each be shearing past each other at different speeds, but the cells all remain nice cubes (e.g. one can safely set the mesh-generating-point velocity in the “x” direction to be an arbitrary function of “y”, \(v^{i}_{\rm mesh,\,x} = f(y_{i})\)). This is illustrated in the cartoon above. Curvilinear grids with circular motion of the meshes (e.g. the DISCO-like behavior described above) are another good example. But if you set “invalid” mesh motion (e.g. \(v^{i}_{\rm mesh,\,x} = f(x_{i})\) in a Cartesian mesh, which could generate arbitrary compression or expansion of neighboring cubical cells), or worse yet allow the cells to move freely with the fluid, you will quickly reach the point where cells cannot possible still be regular (without overlapping or having “holes” between them). At that point the code will give garbage (meaningless) results (or simply crash, if you are lucky). The MFV method in the code, in principle, works with any of these settings, but as described above, you will still introduce quite nasty errors and poor behavior on problems if you choose “bad” mesh motion for your problem.

Note that if HYDRO_FIX_MESH_MOTION is set, the simulation snapshot files will save a field titled ParticleVelocities which includes the motion of the mesh-generating points themselves. This is distinct from the Velocities field, which saves the fluid velocities at the mesh point locations (and is always saved in the snapshots). If HYDRO_FIX_MESH_MOTION is set to values (1), (2), or (3), where the velocities of the mesh-generating points are set in the initial conditions file, then the initial conditions file (or snapshot file, if re-starting from a snapshot) must, of course, contain the field ParticleVelocities with a value specified for all fluid elements (just like the initial masses/fluid velocities/positions/etc.).

5. Similarities & Differences from GADGET (+other codes)

What’s the same as GADGET?

GIZMO is originally descended from the GADGET source code, and owes an incredible debt to GADGET’s creator Volker Springel.

While there are many differences in the code, as much as possible, I’ve tried to keep these “under the hood”. The syntax is very similar to GADGET. The basic operation modes of the code, restarting, code variable names, compiler flags, the setup of the Makefiles and compilers, parameterfiles, parameterfile item names and dimensionless values, etc – many of these should immediately be familiar to anyone who has used GADGET–2 or GADGET–3.

The initial conditions files are identical between GADGET and GIZMO. Anything that worked as a GADGET ICs file, anything that writes out GADGET-format ICs: those ICs can trivially be used as GIZMO ICs with no modification, just as they would for GADGET (and vice versa).

The snapshot formation is also identical. This includes both “GADGET-format” (unformatted binary – almost never the recommended format for simulation snapshots!) and HDF5 format (please use this wherever possible). The only way you’ll see different behavior in a snapshot file is if you compile and run the code with new physics enabled (physics not in GADGET), which writes out additional quantities to the snapshot files that are not included in standard GADGET snapshots.

Long story short, we want to take advantage of the large community using, developing for, and analyzing GADGET simulations: it should be trivial (require essentially nothing changed) to analyze and make ICs for GIZMO simulations instead, it should require almost no extra time to run GIZMO simulations if you’ve run GADGET simulations before, and it should be very easy to port code written for GADGET directly into the GIZMO source.

What’s different?

That said, GIZMO is a totally different code from GADGET “under the hood” in many ways.

Most obviously, the physics options (e.g. magnetic fields, conduction, turbulent diffusion, models for galaxy and star formation, feedback, coupled dust-gas physics, nuclear reaction networks, cooling, exotic dark matter, black holes, etc.) are either completely not present, or are qualitatively totally different from the physics included in GADGET (or in its more direct descendant, AREPO). Where there are some overlaps, they are intentionally included so that we can compare the new code results with the large body of historical simulations. Additionally, GIZMO is now a performance-portable heterogeneous code in which essentially the entire physics integration — hydro chain (density, gradients, hydro force), self-gravity tree (build, LET exchange, walk), cooling/chemistry, nuclear burning, RT, feedback, sinks, AGS, SIDM, and per-particle drift/kick — executes on-device through the Kokkos portability layer (CUDA, HIP, or OpenMP backends), with the host MPI process acting as memory manager and inter-rank dispatcher. The code has been migrated from C to C++ (C++17) to support modern language features, templates, and device-callable inline functions.

But the differences go deeper as well. The actual method for solving the hydrodynamics has almost nothing in common with GADGET. Obviously, if you choose to use a non-SPH hydro solver (e.g. the meshless finite-volume, or lagrangian finite-mass method), you aren’t even using an SPH code anymore! But even in SPH mode, the differences are very fundamental, and include: the functional form of the SPH equations (pressure-energy vs density-entropy), constraints used to determine the SPH smoothing lengths (constant-mass vs constant-particle number in kernel), artificial viscosity (higher-order matrix moments following Cullen and Dehnen’s ‘inviscid SPH’ versus ‘constant AV’), artificial conductivity (not present in GADGET), pairwise symmetric hydro operations (not present in GADGET), high-order integral-based gradient estimation (not present in GADGET), and timestepping (limiter based on Saitoh et al. with a second-order leapfrog, versus unlimited timestepping in single-step marching). Don’t expect to run GIZMO in SPH mode and get exactly the same answer as you would with GADGET!

Even for the N-body code, there have been substantial improvements and revisions to the original GADGET structure. The data structures are somewhat different (for memory reasons), the domain decomposition has been heavily optimized and made more flexible, especially for heterogeneous CPU+GPU runs, the tree walk code has been consolidated, and the gravity solver has been restructured to allow more modular behavior (e.g. additions of external potentials, exotic dark matter or dark energy physics, etc). The entire gravity tree pipeline — domain decomposition, tree build (Karras-style GPU BVH), Local Essential Tree exchange across MPI ranks, and the Barnes-Hut walk itself — now runs on the Kokkos device, with the host MPI process responsible only for inter-rank communication. The gravitational softenings are also now fully adaptive for all particle types (as opposed to fixed). So don’t expect to run GIZMO in even pure N-Body mode and expect to get exactly the same answer as you would with GADGET!

How do you make GIZMO “act like” another code?

But, you ask, what if I want to make GIZMO act more like code X? Well, you shouldn’t (each code is optimized for its own design). But you still want to? Ok, fine, here’s what you can do to get some approximation, for some popular codes:

GADGET: Turn on the compiler flags:

HYDRO_DENSITY_SPH
SPH_DISABLE_CD10_ARTVISC
SPH_DISABLE_PM_CONDUCTIVITY

and turn off (along with all additional fluid physics):

HYDRO_MESHLESS_FINITE_MASS
HYDRO_MESHLESS_FINITE_VOLUME
ADAPTIVE_GRAVSOFT_FORGAS
ADAPTIVE_GRAVSOFT_FORALL

If you’re using cooling/star formation enable only the top-level switches and the flags labeled “old sub-grid models” (no “new” or “PFH” models). Use 32 neighbors, and an artificial viscosity constant=1.

GASOLINE: do everything for as above for GADGET, then enable TURB_DIFF_ENERGY. You should also go into the hydro_core_sph.h file and uncomment the lines under “GASOLINE-like equation of motion” (this will make the equation of motion revert to the Gingold+Monaghan form)

RSPH: be very careful, this method is not conservative and will blow up for all but the most basic test problems. follow the GADGET procedure and then uncomment the lines under “RSPH equation of motion” in hydro_core_sph.h

PHANTOM: our SPH MHD formulation is identical to the most current version from Tricco+Price. The adaptive gravitational softenings for gas follow Price+Monaghan (also similar). For the standard hydro, turn on the flag HYDRO_DENSITY_SPH (and ideally enable the Morris+Monaghan artificial viscosity, but if not, just use the “old” artificial viscosity by turning on SPH_DISABLE_CD10_ARTVISC). Disable the HYDRO_MESHLESS_FINITE_MASS and HYDRO_MESHLESS_FINITE_VOLUME methods. The quintic kernel is optional (but recommended).

SPHS: this one is a bit trickier. At least use the quintic kernel, but ideally you’d insert the same triangular kernel used in SPHS with the much-larger required number of neighbors (they use 442). Follow the instructions otherwise for GASOLINE, except disable TURB_DIFF_ENERGY and enable SPHAV_CD10_VISCOSITY_SWITCH+SPHAV_ARTIFICIAL_CONDUCTIVITY (set the artificial conductivity constant to a high value, >=1).

AREPO: run GIZMO using a Voronoi tesselation to define faces, with the galaxy formation/gas dynamics/cooling physics set to the same as they are above for GADGET. Turn off ADAPTIVE_GRAVSOFT_FORALL.

ATHENA/RAMSES/ENZO/AMR codes: if you wish to emulate a fixed-grid code, you can run GIZMO in fixed-mesh mode, and set up your ICs appropriately, to mimic the functionality of these methods. However, refinement or particle-splitting in GIZMO is only implemented for the Lagrangian methods (the regular grid method are not coded to work with an irregular, but strictly Cartesian non-moving grid) as one obtains with AMR, so if that functionality is desired, you need to run RAMESES/ENZO/other AMR codes (sorry)

6. Compiling and Running the Code (a brief tutorial)

Compilation requirements

First, get a copy of the source code from the code repository.

You should follow the how-to guides from the websites telling you how to use and pull code from the Git repository (allowing you to see all versions, fork your own for modifications without interfering with the main branch, and merge changes/updates in an orderly manner).

The code requires a few common libraries to run. On most clusters, these are considered standard and can be loaded through some sort of module system. Consult your local sysadmin or the user guide for your machine for details. For many of the computers at national centers (or if you’re at any of the schools where many of our other users are located), the Makefile within the source code will already have notes specifically telling you how to load the modules for that machine. For example, on the TACC “Frontera” machine, you just need to include the line “module load intel impi hdf5 fftw3” in your .bashrc or .bash_profile file (or enter it into the command line before compiling), then link to the libraries in the Makefile as is currently done (open the Makefile and search “Frontera” to see these instructions – there are similar instructions for all other pre-programmed machines within the Makefile, so take a look).

The necessary libraries are: MPI, FFTW (version 3), and HDF5:

The GIZMO code package includes various .c and .h files, folders with different modules, and the following critical files:

Starting and Running the Code

To start a simulation, invoke the executable with a command like

mpirun -np 32 ./GIZMO myparameterfile.param

This will have to be modified for the machine you’re using (“mpirun” may have a different syntax, but should be a completely standard MPI call: see the users’ guide for whatever system you’re on for the details). This example would run the simulation with 32 processors, and with simulation parameters as specified in the parameter file of name myparameterfile.param (discussed in detail on its own page).

The code does not need to be recompiled for a different number of processors, or for a different problem size. It is necessary to recompile if you are using the code in hybrid OPENMP/MPI (multi-threaded) mode, and want to change the number of threads (processors) per MPI process. Note that there is no formal requirement for the processor number to be a power of two (though that can sometimes be most efficient for communication). While GIZMO runs, it will print out various log-messages that inform you about the code activity. If you run a simulation interactively (as in the above call), these log-messages will appear on the screen, but you can also re-direct them to a file.

For normal runs at any cluster computer, you will usually have to put the “mpirun” comment into a script file submitted to the computing queue (again, see your computers users guide) – in this case, it will automatically pipe standard output/error messages to files, but you may still want to specify filenames in the script. This is required for the queue systems to organize the jobs, though you can usually put in special requests to run jobs in interactive mode.

Here is a typical example SLURM script (most modern systems use this or PBS as their job submission system): 

#!/bin/bash  
#SBATCH -J TEST -p normal -N 100 --ntasks-per-node 16 -t 01:00:00 -A ALLOCATIONNAME  
export OMP_NUM_THREADS=2  
source $HOME/.bashrc  
module purge  
module load intel impi hdf5 fftw3  
ibrun ./GIZMO ./params.txt 0 1>gizmo.out 2>gizmo.err

This is a script submitting job-name TEST, requesting it go in the NORMAL queue, run on 100 nodes, with 16 MPI tasks per node, running for 1 hour (time in HH:MM:SS format), charged to allocation ALLOCATIONNAME. We’ve set it to use 2 OPENMP threads. We’ve also used the module system of the machine to load the relevant shared libraries (intel compiler, intel-MPI, hdf5, and fftw here). We can also load the modules through our personal .bashrc file, so including both calls here is a bit redundant. Then we submit the job, using ibrun (this is like mpirun above: different compilers and machines have different calls for running MPI executables), to call our compiled GIZMO executable in the local directory, with parameterfile params.txt in the same directory, restartflag 0 (start from ICs). The 1>gizmo.out 2>gizmo.err are standard bash prompts that redirect stdout and stderr to files with those names, respectively (otherwise the machine will decide their default names, which you may prefer). Note that on different machines, the modules will be different, as will the ibrun/mpirun call, as will some of the required flags. Some machines will use #PBS instead of #SBATCH. You need to read the machine user guide to know how to submit on a particular machine. Also read the SBATCH or PBS (whichever you are using) manual page to learn what all the different flag options are. Finally, you could name this script something like runscript and submit it with the command sbatch runscript.

Interrupting a run

Run interruption works just like in GADGET. From that users guide:
Usually, a single submission to a queue system will not provide enough CPU-time to process a big production run. Therefore, a running simulation can be interrupted after any timestep, and resumed at the very same place later on. If the CPU-time limit is specified correctly in the parameter file, the code will interrupt itself automatically before the CPU-time limit expires, and write a set of ‘restart’-files. Actually, each processor writes its own restart file. These restart-files can be used to continue the run conveniently (see below).
Sometimes you may want to interrupt the code manually with the possibility to resume it later on. This can be achieved by generating a file named “stop” in the output-directory of a simulation, e.g.

cd /active_run_directory/  
> stop

The code will then write a restart-file and terminate itself after the current timestep is completed, and the file ‘stop’ will be erased automatically. Restart files are also generated when the last timestep of a simulation has been completed. They can be used if one later wants to extend the simulation beyond the original final time.

Performance-Portable Heterogeneous Execution (CPU-GPU, Kokkos)

GIZMO is a flexible, performance-portable heterogeneous code: the same source tree targets NVIDIA GPUs (via CUDA), AMD GPUs (via HIP), and multi-core CPU-only nodes (via OpenMP), selected at compile time through the Kokkos portability library and the Makefile machine configuration. The design philosophy is that almost all numerically-intensive work runs on the device. On a GPU node the device is the GPU; on a CPU-only Kokkos build the device is the OpenMP thread pool — but in either case the same kernels execute, with the host MPI process acting primarily as a memory manager, dispatcher, and inter-rank communicator. Builds without Kokkos are no longer supported.

What runs on the device

Practically every per-particle, per-pair, per-node, and per-substep loop is now device-resident. This includes not just the kernel evaluations themselves, but the full surrounding pipeline — neighbor-list construction, kernel calculations, post-processing reductions, tree construction, tree walks, drift/kick/timestep updates, and (where applicable) domain-decomposition work all execute on the device. The host is responsible for MPI traffic, file I/O, the high-level time-loop, and the small amount of bookkeeping where collective decisions or rank-to-rank sends/receives are unavoidable. The current device-resident inventory:

The host MPI process retains responsibility for: rank-to-rank communication (ghost exchange Alltoallv, LET Iallgatherv, domain rebalancing collectives), I/O (snapshots, restart files, on-the-fly group catalogs), the global time-step decision tree, and a handful of small reductions (energy, momentum, mass conservation diagnostics) where the cost of a device kernel would exceed the cost of a host loop. In a typical step on a GPU node, the host wall-time is dominated by MPI and I/O — almost the entire physics integration happens on the GPU.

Numerically, all device-resident kernels produce results that are bitwise-identical to the corresponding host-only run on the same hardware, and within floating-point summation-order tolerance across CPU/GPU. The GPU gravity tree walk and GPU hydro chain have been validated to bitwise identity against the (now-retired) legacy CPU tree-walk reference at 1- and 2-rank scale on both NVIDIA GH200 (Vista) and Apple-Silicon Kokkos-OpenMP builds (MacBookCellar_Kokkos).

How to enable

Performance-portable execution is always active when Kokkos is linked, which is controlled by SYSTYPE in the Makefile (not by any Config.sh flag). The neighbor-list infrastructure, ghost exchange, GPU kernel dispatch, persistent particle arena, GPU gravity tree, and GPU domain decomposition are all unconditionally enabled in any Kokkos build. The choice between GPU and CPU-OpenMP execution is determined entirely by the Kokkos backend selected at Kokkos build time:

The Makefile must link to Kokkos and (for GPU) the appropriate device compiler. See the Vista (Vista GH200, CUDA), MacBookCellar_Kokkos (Apple Silicon, OpenMP), or Frontier (AMD MI250X, HIP) blocks in the Makefile for working examples. On HPC systems where Kokkos is installed as a module (module load kokkos), the include and library paths are typically set by environment variables. There are no Config.sh flags that toggle GPU vs CPU dispatch — both code paths are always compiled in, and the Kokkos backend selects which one runs.

Architecture

The implementation follows an “AthenaK-style” pattern: device-callable physics functions live in .h header files (annotated KOKKOS_INLINE_FUNCTION), while .cc files contain only host-side orchestration — memory management, MPI communication, and Kokkos::parallel_for launches. A __managed__ copy of the global All parameter struct (All_dev) provides device kernels with read-only access to simulation parameters. Each device translation unit (e.g. density_gpu.cc, cooling.cc, gravity/gpu_gravtree.cc) is compiled by the device compiler (nvcc/hipcc/host C++) and linked with the rest of the code.

A typical timestep, with all heavy work on the device, looks roughly like:

  1. Domain decomposition (device): Peano-Hilbert key build, GPU topology build/finalize, work-cost balancing.
  2. Drift (device): per-particle position/velocity predictor over the active set.
  3. Tree build (device): Morton sort + Karras-style BVH construction; topnode moment resummation; incremental moment refresh on subsequent substeps via dirty-node tracking.
  4. LET exchange (host MPI + device pack/unpack): each rank packs its locally-essential tree on device, exchanges via MPI_Iallgatherv, and integrates foreign tree segments into the device-resident SoA.
  5. Gravity walk (device): speculative Barnes-Hut walk for the active set; force, optional potential, tidal tensor, jerk; adaptive softening; PMGRID short-range table. The walk is “speculative” in the sense that all leaves are descended optimistically with the opening criterion checked at the multipole level (rather than partitioning the tree into pre-decided open/closed nodes on the host first); this maps efficiently to GPU warps/wavefronts where divergent control flow is expensive, at the cost of doing slightly more work per particle compared to a host-side adaptive walker. The net result is a substantial wall-clock speedup over the legacy export-loop CPU tree walk, especially for full-active steps where every particle’s walk runs concurrently.
  6. Ghost exchange (host MPI): import boundary gas particles from neighboring ranks via tile-based MPI_Alltoallv into the device-resident ghost arena.
  7. Hydro chain (device): density h-convergence → symmetric neighbor list build → gradient reconstruction → Riemann/flux hydro force.
  8. Ghost writeback (host MPI): reverse-communicate j-particle modifications to home ranks.
  9. Source/subgrid physics (device): cooling, chemistry, RT subcycle, mechanical/thermal/radiative feedback, sinks, AGS, SIDM, etc.
  10. Kick + timestep update (device).

The persistent device arena means that between these stages, particle data does not move — P[] and CellP[] are read and written in place by successive device kernels with no host-device deep-copies.

Performance characteristics

Headline results on NVIDIA GH200 (Vista, single GPU, Kokkos-CUDA) versus the same Vista node running the CPU-only reference build (SYSTYPE=Vista_CPU, GCC, no Kokkos):

Test Particles CPU (s/step) GPU (s/step) Speedup
poisson_box (hydro + gravity) 125k 1.92 0.27 7.2×
poisson_box (hydro + gravity) 1M 17.0 2.14 7.9×
gmc_cooling (MHD + cooling + SF) 512k 25.8 4.8 5.4×

Per-phase speedups for gmc_cooling 512k: hydro force kernel ~32×, gradient ~6.7×, cooling ~3.5×, density ~3.2×. The hydro pipeline as a whole runs ~7× faster end-to-end. Cooling has the smallest relative speedup because the underlying solver is already well-vectorized on CPU and the implicit chemistry network is short.

At small particle counts (≲100k per rank), Kokkos kernel-launch overhead and CSR-build overhead can dominate and the GPU build is no faster — sometimes slightly slower — than CPU-OpenMP. Speedup grows with N and dominates the total wall-time at production scales. Ghost-exchange and LET MPI traffic remain on the host and are unaffected by the device backend; at very large rank counts these can become the bottleneck, motivating the “host-as-MPI-only” mental model.

Restarting a run

Restarting from restart-files

This also functions just like GADGET–2: Quoting that users guide: To resume a run from restart-files, start the code with an optional flag in the form

mpirun -np 32 ./GIZMO myparameterfile.param 1

This is just the normal run call with the flag “1” after the parameterfile name. This will continue the simulation with the set of restart files in the output-directory, with the latter being specified in the parameterfile. Restarting in this fashion is transparent to the code, i.e. the simulation behaves after the restart exactly as if had not been interrupted to begin with.

When the code is started with the restart-flag, the parameterfile is parsed, but only some of the parameters are allowed to be changed, while any changes in the others will be ignored.

It is important to not forget the 1 if you want to restart – otherwise the simulation will restart from scratch, i.e. by reading in the initial conditions again! Also note that upon restarting from restart-files, the number of processors used for a simulation cannot be changed. Also note that restart files can in general not be transferred to another system, and that their structure is completely different than that of snapshot files. If you want to continue a simulation on another system or with a different number of processors, restarting from a snapshot file is the method of choice. This will be discussed next.

Restarting from snapshot-files

This also works just like GADGET–2: There are two possibilities to restart a simulation from a previous snapshot-file.

In the first possibility, one simply adopts the snapshot file as new initial conditions. Note that this option requires changes in the parameterfile: You need to specify the snapshot-file as initial- conditions-file (and set ICFormat to match the format of this new ICs file), you also need to set TimeBegin (see below) to the correct time corresponding to the snapshot-file if it is a cosmological run (otherwise it will begin with the wrong redshift; for non-cosmological runs the zero-point of time is arbitrary so this won’t make any meaningful difference). In addition, you should change the base filename for the snapshot files, since the counter for the outputs will start at 0 again, thereby possibly overwriting outputs you might already have obtained. Once this is done, you can continue the run from the snapshot-file (without the optional 1).

An alternative to this somewhat inconvenient procedure is to restart the code with a flag equal to 2 after the name of the parameterfile, i.e. just like above for the restart from restart-files, but with the 1 replaced by 2. In this case, you only have to change the name of the initial conditions file and make it equal to the name of the snapshot you want to start from, other parameters in the parameterfile need not to be changed. In particular, the code will continue the numbering of all further snapshots starting with 1 plus the number of the snapshot that is used as input file. When you specify a snapshot file that was distributed onto several files (e.g. dump 007.0 to dump 007.7) you should only give the base-name (i.e. here dump 007) as initial conditions file when using this option.

Note that the restart from snapshot files allows a change of the number of processors used for the simulation. This is not possible if you restart from restart-files. However, restarting from restart-files is the preferred method to resume a simulation, because it is much faster, and it minimizes possible perturbations in the time integration scheme of a running simulation (restarting from a snapshot forces the individual timestep scheme to be resynchronized). Because in general not all the particles are synchronized at the time of writing a snapshot file (the others are first-order predicted to the output time), a small perturbation in the time integration is introduced when restarting from a snapshot file.

7. Code Modules in Config.sh (Setting compile-time options, aka physics and methods)

Overview

Many options in GIZMO are controlled with compile-time options in the file “Config.sh” rather than run-time options in the parameterfile. This is done, following GADGET–2, in order to allow the generation of highly optimized binaries by the compiler, even when the underlying source allows for many different ways to run the code. Unfortunately, this technique has the disadvantage that different simulations may require different binary executables. If several simulations are run concurrently, there is hence the danger that a simulation is started or resumed with the ‘wrong’ binary. You should therefore always produce a separate executable for each simulation that is run. A good strategy for doing this in practice is to make a copy of the whole simulation source code together with its makefile in the output directory of each simulation run, and then use this copy to compile the code and to run the simulation. The code and its settings become then a logical part of the output generated in the simulation directory. – (V. Springel). (Trust me, if you don’t do this, it will quickly become impossible to know what version of the code was used to run any simulation – PFH).

The Template_Config.sh file contains a dummy list of all available compile-time code options (with short descriptions), with most of them commented out by default. To activate a certain feature, the corresponding parameter should be un-commented (and given the desired value, where appropriate) and copied into “Config.sh”.

The Template_Config.sh also contains critical information (also listed below, but check both) about which modules are still in development, and who their developers are: please contact them before using such a module (see Notes on Stable vs. Development Branches). Please follow the citation guidelines and cite the relevant code papers whenever you use a given module.

Whenever you change one of the makefile options described below, a full recompilation of the code is necessary: it must be done “from scratch” so you have to type “make clean” in the source directory followed by “make”.

Below, we describe the entire contents of the Template_Config.sh file, in groups based on their type.

Boundary Conditions & Dimensions

For almost any problem, you need to specify things like boundary conditions and how many dimensions the problem is being solved in. So these options are potentially always relevant.

#################################################################################################### 
#--------------------------------------- Boundary Conditions & Dimensions 
#################################################################################################### 
#BOX_SPATIAL_DIMENSION=3    # sets number of spatial dimensions evolved (default=3D). Switch for 1D/2D test problems: if =1, code only follows the x-line (all y=z=0), if =2, only xy-plane (all z=0). requires SELFGRAVITY_OFF
#BOX_PERIODIC               # Use this if periodic/finite boundaries are needed (otherwise an infinite box [no boundary] is assumed)
#BOX_BND_PARTICLES          # particles with ID=0 are forced in place (their accelerations are set =0): use for special boundary conditions where these particles represent fixed "walls"
#BOX_SHEARING=1             # shearing box boundaries: 1=r-z sheet (r,z,phi coordinates), 2=r-phi sheet (r,phi,z), 3=r-phi-z box, 4=as 3, with vertical gravity
#BOX_SHEARING_Q=(3./2.)     # shearing box q=-dlnOmega/dlnr; will default to 3/2 (Keplerian) if not set
#BOX_LONG_X=140             # modify box dimensions (non-square finite box): multiply X (not compatible with periodic gravity: if BOX_PERIODIC or PMGRID is active, make sure SELFGRAVITY_OFF or GRAVITY_NOT_PERIODIC is on)
#BOX_LONG_Y=1               # modify box dimensions (non-square finite box): multiply Y
#BOX_LONG_Z=1               # modify box dimensions (non-square finite box): multiply Z
#BOX_REFLECT_X=0            # make the x-boundary reflecting (assumes a box 0<x<BoxSize_X, where BoxSize_X=BoxSize*BOX_LONG_X, if BOX_LONG_X is set); if no value set or =0, both x-boundaries reflect, if =-1, only lower-x (x=0) boundary reflects, if =+1, only upper-x (x=BoxSize) boundary reflects
#BOX_REFLECT_Y              # make the y-boundary reflecting (assumes a box 0<y<BoxSize_Y); if no value set or =0, both y-boundaries reflect, if =-1, only lower-y (y=0) boundary reflects, if =+1, only upper-y (y=BoxSize) boundary reflects
#BOX_REFLECT_Z              # make the z-boundary reflecting (assumes a box 0<z<BoxSize_Z); if no value set or =0, both z-boundaries reflect, if =-1, only lower-z (z=0) boundary reflects, if =+1, only upper-z (z=BoxSize) boundary reflects
#BOX_OUTFLOW_X=0            # make the x-boundary outflowing (assumes a box 0<x<BoxSize_X, where BoxSize_X=BoxSize*BOX_LONG_X, if BOX_LONG_X is set); if no value set or =0, both x-boundaries outflow, if =-1, only lower-x (x=0) boundary outflows, if =+1, only upper-x (x=BoxSize) boundary outflows
#BOX_OUTFLOW_Y              # make the y-boundary outflowing (rules follow BOX_OUTFLOW_X, for the y-axis here). note that outflow boundaries are usually not needed, with Lagrangian methods, but may be useful in special cases.
#BOX_OUTFLOW_Z              # make the z-boundary outflowing (rules follow BOX_OUTFLOW_X, for the z-axis here)
####################################################################################################

These options determine basic aspects of the boundary conditions and dimensionality of the problem. Because these completely change the nature of the solvers and neighbor searches, they must be set at compile (not run) time.

BOX_SPATIAL_DIMENSION: This determines how many dimensions will actually be evolved for the simulation. If not set, defaults to 3 (3D x,y,z). If set =1, the simulation will be one-dimensional, and only follow the x-line, with all \(y=z=0\) (\(y=z=0\) should be set in the initial conditions, or it may cause the code to crash). If set =2, the simulation will be two-dimensional, and only follow the x-y plane, with all \(z=0\) (again, make sure \(z=0\) is set in the initial conditions). Higher-than-three dimensional systems are not implemented (sorry, no hyper-cubes). Lower-dimensional integration only works with SELFGRAVITY_OFF (gravity off, or the radiation version of this).

BOX_PERIODIC: set this if you want to have periodic boundary conditions.

BOX_BND_PARTICLES: If this is set, particles with a particle-ID equal to zero do not receive any hydrodynamic acceleration. This can be useful for idealized tests, where these particles represent fixed ‘walls’. They can also be modified in kicks.c to give reflecting boundary conditions.

BOX_SHEARING: This enables shearing-box boundary conditions. It should be set to different values corresponding to the type of shearing box. (=1) is a 2D un-stratified shearing-sheet in the r-z plane. note that if this is set, the output coordinates (0,1,2) correspond to (r,z,phi). (=2) is a 2D shearing sheet in the r-phi plane; coordinates (0,1,2)=(r,phi,z). (=3) is a full 3D box, (0,1,2)=(r,phi,z). (=4) is the full box but vertically stratified (with vertical gravity).

BOX_SHEARING_Q: This sets the “q” parameter for shearing boxes. q=-dln(Omega)/dln(R), =3/2 for a Keplerian disk or =1 for a constant-Vc disk. If this isn’t set but BOX_SHEARING is, it will default to =3/2 (Keplerian).

BOX_LONG_X/Y/Z: These options can be used when the boundary conditions make the box finite in extent () When set, they make the periodic simulation box rectangular with dimensions BOX_LONG_? times BoxSize in each of the ?=X,Y,Z axes.

BOX_REFLECT_X/Y/Z: This makes the boundaries reflecting: set BOX_REFLECT_? with the appropriate ?=X,Y,Z to make the given X, Y, or Z axes reflecting. If the box is ‘open’, then this will still assume the ‘reflecting domain’ to be effectively a box that stretches from location x,y,z = 0 to x,y,z = BoxSize_x,y,z (the value of the BoxSize parameter times any appropriate BOX_LONG multipliers you may have enabled). If you have enabled periodic boundaries with BOX_PERIODIC, then this will override the periodic boundary in whatever axes you make reflecting. If you simply enable this without specifying a numerical value, or give it the value =0 (e.g. BOX_REFLECT_X or BOX_REFLECT_X=0) then it will set both axis along the appropriate dimension to be reflecting (i.e. both the ‘lower’ x=0 and ‘upper’ x=BoxSize_x boundary will be reflecting). If you set it to the value =-1 (e.g. BOX_REFLECT_X=-1), then only the ‘lower’ (x=0) boundary along the relevant dimension will be reflecting. If you set it to the value =1 (e.g. BOX_REFLECT_X=1), then only the ‘upper’ (x=BoxSize_x) boundary along the relevant dimension will be reflecting. Currently, reflecting boundaries are only implemented for the quasi-Lagrangian code methods (not fixed-grids).

BOX_OUTFLOW_X/Y/Z: This makes the boundaries outflowing – any elements which flow through a given boundary will simply be deleted from the simulation domain. The syntax here is identical to that of the BOX_REFLECT flags described above. Set BOX_OUTFLOW_? with the appropriate ?=X,Y,Z to make the given X, Y, or Z axes outflowing. If the box is ‘open’, then this will still assume the ‘outflow boundary’ corresponds to either the ‘lower’ axis at x,y,z = 0 or the ‘upper’ axis at x,y,z = BoxSize_x,y,z (the value of the BoxSize parameter times any appropriate BOX_LONG multipliers you may have enabled). If you have enabled periodic boundaries with BOX_PERIODIC, then this will override the periodic boundary in whatever axes you make outflowing. If you simply enable this without specifying a numerical value, or give it the value =0 (e.g. BOX_OUTFLOW_X or BOX_OUTFLOW_X=0) then it will set both axis along the appropriate dimension to be outflowing (i.e. both the ‘lower’ x=0 and ‘upper’ x=BoxSize_x boundary will be outflowing). If you set it to the value =-1 (e.g. BOX_OUTFLOW_X=-1), then only the ‘lower’ (x=0) boundary along the relevant dimension will be outflowing. If you set it to the value =1 (e.g. BOX_REFLECT_X=1), then only the ‘upper’ (x=BoxSize_x) boundary along the relevant dimension will be outflowing. Currently, outflow boundaries are only implemented for the quasi-Lagrangian code methods (not fixed-grids). Note that in many circumstances, there is no actual need for a formal outflow boundary condition – since the code is Lagrangian, resolution elements can simply expand outwards in e.g. simulations with outflows to infinity, and if the densities are low at infinity, their timesteps will usually be long as well (making it negligible in CPU cost to ‘retain’ said elements). But in some cases, this can be useful for either physical or numerical reasons.

Fluid Solver Method

These options control the fluid solver. Note that this applies to almost anything in the code being treated as a fluid so not just hydrodynamics, but hydrodynamics, MHD, radiation in many of the radiation transport modules, cosmic rays in the distribution function limit, some self-interacting dark matter models, and many more. So again, this is usually relevant, though there are also of course cases (like a pure N-body gravity simulation) where there is actually no fluid dynamics in the simulation, in which case you can safely ignore these flags entirely. 

####################################################################################################
# --------------------------------------- Hydro solver method
####################################################################################################
# --------------------------------------- Finite-volume Godunov methods (choose one, or SPH)
#HYDRO_MESHLESS_FINITE_MASS     # solve hydro using the mesh-free Lagrangian (fixed-mass) finite-volume Godunov method
#HYDRO_MESHLESS_FINITE_VOLUME   # solve hydro using the mesh-free (quasi-Lagrangian) finite-volume Godunov method (control mesh motion with HYDRO_FIX_MESH_MOTION)
#HYDRO_REGULAR_GRID             # solve hydro equations on a regular (recti-linear) Cartesian mesh (grid) with a finite-volume Godunov method
## -----------------------------------------------------------------------------------------------------
# --------------------------------------- Options to explicitly control the mesh motion (for use with the MFV or grid solvers): only set for non-standard behavior
#HYDRO_FIX_MESH_MOTION=0        # mesh with arbitrarily-defined mesh-generating velocities: (0=non-moving, 1=fixed-v [set in ICs] cartesian, 2=fixed-v [ICs] cylindrical, 3=fixed-v [ICs] spherical, 4=analytic function, 5=smoothed-Lagrangian, 6=glass-generating, 7=fully-Lagrangian)
#HYDRO_GENERATE_TARGET_MESH     # use for IC generation (can be used with -any- hydro method: MFM/MFV/SPH/grid): this allows you to specify in the functions 'return_user_desired_target_density' and 'return_user_desired_target_pressure' (in eos.c) the desired initial density/pressure profile, and the code will try to evolve towards this.
## -----------------------------------------------------------------------------------------------------
# --------------------------------------- SPH methods (enable one of these flags to use SPH):
#HYDRO_PRESSURE_SPH             # solve hydro using SPH with the 'pressure-sph' formulation ('P-SPH')
#HYDRO_DENSITY_SPH              # solve hydro using SPH with the 'density-sph' formulation (GADGET-2 & GASOLINE SPH)
# --------------------------------------- SPH artificial diffusion options (use with SPH; not relevant for Godunov/Mesh modes)
#SPH_DISABLE_CD10_ARTVISC       # for SPH only: Disable Cullen & Dehnen 2010 'inviscid sph' (viscosity suppression outside shocks); just use Balsara switch
#SPH_DISABLE_PM_CONDUCTIVITY    # for SPH only: Disable mixing entropy (J.Read's improved Price-Monaghan conductivity with Cullen-Dehnen switches)
## -----------------------------------------------------------------------------------------------------
# --------------------------------------- Kernel Options
#KERNEL_FUNCTION=3              # Choose the kernel function (2=quadratic peak, 3=cubic spline [default], 4=quartic spline, 5=quintic spline, 6=Wendland C2, 7=Wendland C4, 8=2-part quadratic, 9=Wendland C6)
#KERNEL_CRK_FACES               # Use the consistent reproducing kernel [higher-order tensor corrections to kernel above, compared to our usual matrix formalism] from Frontiere, Raskin, and Owen to define the faces in MFM/MFV methods. can give more accurate closure, potentially improved accuracy in MHD problems. remains experimental for now.
####################################################################################################

These options determine which hydro solver is used (see the section in this guide about the differences between the different hydro methods). Make sure you enable ONLY ONE hydro method labeled HYDRO_... (i.e. don’t enable both HYDRO_MESHLESS_FINITE_MASS and HYDRO_MESHLESS_FINITE_VOLUME or HYDRO_PRESSURE_SPH, etc.), or the code will not behave correctly.

You should read the section on Fluid (Hydro) Solvers for more details on these, before choosing any of these options.

HYDRO_MESHLESS_FINITE_MASS: Use the MFM (Meshless Finite Mass) method to solve hydro. As discussed in the “Hydro Solver” section, if this is set, the mesh must move with the fluid, so any setting for HYDRO_FIX_MESH_MOTION will be ignored.

HYDRO_MESHLESS_FINITE_VOLUME: Use the MFV (Meshless Finite Volume) method to solve hydro. This can be used with a variety of options for the mesh motion set by HYDRO_FIX_MESH_MOTION.

HYDRO_REGULAR_GRID: Assume a regular (recti-linear, Cartesian) grid to solve hydro. If this method is used, the initial conditions must have the gas elements laid out in a regular Cartesian mesh with the same number of dimensions as the code is set to run, and the neighbor number must be set large enough that on a regular grid it would include at least the nearest neighbors in each cardinal direction – otherwise it cannot find these neighbors. The kernel-based methods are still used for gradient calculation, as these are more flexible for higher-order gradient estimation, so it is good to use ‘normal’ kernel numbers (somewhat more than the number of immediate cardinal neighbors – effectively a larger stencil). As discussed in the “Hydro Solver” section, this can be used with a limited sub-set of the mesh-motion options, provided they preserve the rectilinear grid.

HYDRO_FIX_MESH_MOTION: This flag allows you to explicitly control the motion of the mesh, to obtain non-default behavior. It does not need to be set for any method (but unlike the other HYDRO_... flags, it can be set in concert with a hydro method). Details are described in the “Hydro Solver” methods section above. If you use a fixed-mass method (MFM or SPH), this will be ignored, because the mesh motion is only allowed to be with the fluid. If you use MFV the default code behavior is to move with the locally-smoothed velocity field. If you use regular grids, the default behavior is non-moving grids. But setting this with MFV or regular grids allows you to specify arbitrary mesh motion, for example shearing, rotating, expanding/contracting boxes.

HYDRO_GENERATE_TARGET_MESH: This flag is designed for building initial conditions and equilibrium objects for simulations. It can be used with any hydro method. If enabled, you must provide the functions return_user_desired_target_density and return_user_desired_target_pressure, which are defined in the very top of the eos/eos.c file. The code will then attempt to evolve towards these ICs. This is accomplished by re-setting the pressure (and corresponding internal energies) in each timestep to a value \(P = P_{target}(\rho/\rho_{target})\) – in other words, if the density is locally above/below the target value, it will over/under-pressurize the region to move the particles. This is described in more detail in the Initial Conditions (Making & Reading Them) section below. Note that if you set up your target pressure to have gradients, there must be a restoring force designed to balance them (e.g. a gravitational force, which can be solved via self-gravity or imposed analytically with the appropriate parameter choices). Also note this will obviously generate some motion of the mesh; if you wish that to be damped, to arrive at something non-moving, find the line with this compiler flag in kicks.c and un-comment it (that will enable velocity damping).

HYDRO_PRESSURE_SPH: Use SPH to solve hydro, specifically the Lagrangian “pressure-energy” formulation from Hopkins 2013. This will move the gas cell/particles with the fluid, so any HYDRO_FIX_MESH_MOTION setting is ignored.

HYDRO_DENSITY_SPH: Use SPH to solve hydro, specifically the Lagrangian “density” formulation from Springel and Hernquist 2002. This will move the gas cell/particles with the fluid, so any HYDRO_FIX_MESH_MOTION setting is ignored.

SPH_DISABLE_CD10_ARTVISC: This option is relevant if and only if you are using one of the SPH methods. If turned on, it will disable the matrix-based higher-order gradient switch from Cullen & Dehnen 2010 for the SPH artificial viscosity (a switch to attempt to “turn down” artificial viscosity away from shocks). Instead, the older Morris 1987 constant artificial viscosity + Balsara switch method (as in GADGET) is used.

SPH_DISABLE_PM_CONDUCTIVITY: This option is relevant if and only if you are using one of the SPH methods. If turned on, it will disable Price & Monaghan 2008-like “artificial conductivity” – an artificial thermal diffusion term designed to smear out contact discontinuities and shocks in SPH so that SPH can better treat them and ensuing fluid instabilities (at the cost of substantially increased diffusion and loss of entropy conservation).

KERNEL_FUNCTION: This option allows you to replace the standard cubic spline kernel (used for the volume partition with any of the mesh-free methods, and for gravitational softening in any method) with alternative kernel functions (all defined in the file kernel.h). Kernels currently implemented include: 1=linear ramp (\(\sim(1-r)\)); this is not generally recommended (it has certain well-known numerical problems), but is useful for some testing purposes. 2=quadratic ‘peaked’ kernel (\(\sim (1-r)^{2}\)). This is use-able for MFM/MFV methods, but should not be used for SPH, since it does not have a well-defined derivative at r=0 (the kernel derivative is a fundamental quantity in SPH, but meaningless in MFM/MFV). Still it is more noisy than higher-order kernels. 3=default (cubic spline). The recommended number of neighbors for the cubic spline is ~32 (in 3D). 4=Morris 1996 quartic spline. The recommended number of neighbors is ~45–62 (in 3D). 5=Morris 1996 quintic spline. Recommended ~64–128 neighbors (in 3D). 6=Wendland C2 kernel (see Cullen & Dehnen 2010 for details of this kernel). Recommend 3D 45–80 neighbors. In SPH, this kernel produces somewhat larger zeroth-order errors in density estimation compare to the quartic spline, but does not suffer the particle pairing instability. 7=Wendland C4 kernel (64–180 3D neighbors), 8=quadratic 2-step kernel (commonly used in image filtering, and has some special applications here, but not generally as accurate for fluid dynamics; 30–64 neighbors), 9=Wendland C6 kernel (180–600 neighbors). Higher-order kernels are more expensive and more diffusive. They are usually not necessary if MFM-mode, or MFV-mode is used. But if SPH is used, convergence requires (technically) infinite neighbor number. So even though this is more expensive, and more diffusive, it is the only way to suppress certain zeroth and first-order inconsistency error terms which appear in all SPH formulations.

KERNEL_CRK_FACES: Use the consistent reproducing kernel [higher-order tensor corrections to kernel above, compared to our usual matrix formalism] from Frontiere, Raskin, and Owen to define the faces in MFM/MFV methods. Enabling this will augment the usual kernel function (which should still be defined as usual with KERNEL_FUNCTION) with the tensor-corrected consistent reproducing kernel as defined in Frontiere, Raskin, and Owen (arXiv:1605.00725). This is currently valid for MFM/MFV. The CRK kernel gives an effective ‘face’ which is corrected to one-higher-order accuracy, compared to the default matrix-based face elements defined for MFM/MFV. This comes at some cost in memory and extra computations in the gradient step. However because of the higher-order correction, this gives more accurate ‘closure’ relations for the effective faces, which may reduce certain errors: it reduces numerical diffusion in multi-phase boundaries, allows for better maintaining of irregularly-shaped and sharp contact discontinuities, and most obviously reducing div-B errors in MHD. However it remains in testing/experimental stages right now, so please experiment for yourself.

Additional Fluid Physics

####################################################################################################
# --------------------------------------- Additional Fluid Physics
####################################################################################################

These options set different fluid physics. This includes changing the equation of state, treating magnetic fields with ideal or non-ideal MHD, enabling explicit conduction, viscosity, and passive scalar diffusion, modeling turbulent eddy diffusivity, explicit integration of aerodynamic particles in fluid or relativistic (“cosmic ray”) fluids, and so on.

Equations-of-State

##-----------------------------------------------------------------------------------------------------
#---------------------------------------- Gas Equations-of-State
#MEAN_MOLECULAR_WEIGHT_DEFAULT=(2.3)  # Mean molecular weight assumed wherever no chemistry is solved (defaults to the fully-ionized 0.59); set to e.g. 2.3 for a run whose gas is cold and molecular
#EOS_GAMMA=(5.0/3.0)            # Polytropic Index of Gas (for an ideal gas law): if not set and no other (more complex) EOS set, defaults to EOS_GAMMA=5/3
#EOS_HELMHOLTZ                  # Use Timmes & Swesty 2000 EOS (for e.g. stellar or degenerate equations of state); if additional tables needed, download at http://www.tapir.caltech.edu/~phopkins/public/helm_table.dat (or the GitHub site)
#EOS_TILLOTSON                  # Use Tillotson (1962) EOS (for solid/liquid+vapor bodies, impacts); custom EOS params can be specified or pre-computed materials used. see User Guide and Deng et al., arXiv:1711.04589
#EOS_ANEOS                      # Use ANEOS/SESAME tabulated EOS (for planetary impacts/collisions); specify table file paths in parameterfile (AneosTable0, AneosTable1, etc.)
#EOS_ELASTIC                    # treat fluid as elastic or plastic (or visco-elastic) material, obeying Hooke's law with full stress terms and von Mises yield model. custom EOS params can be specified or pre-computed materials used.
#PLANET_HEATING                 # radiogenic + accretional heating of solid bodies (auto-enables EOS_TILLOTSON)
## 
#---------------------------------------- Custom EOS's useful for certain problems
#EOS_SUBSTELLAR_ISM             # EOS for atomic-molecular mixtures (Vaidya+15) for star formation
#EOS_GMC_BAROTROPIC             # idealized barytropic EOS from Masunaga+Inutsuka 2000.
----------------------------------------------------------------------------------------------------

MEAN_MOLECULAR_WEIGHT_DEFAULT: Sets the mean molecular weight assumed wherever no chemistry is actually being solved. The default is the fully-ionized value 0.59; set it to e.g. 2.3 for a run whose gas is cold and molecular, where that default would badly mis-state the temperature corresponding to a given internal energy.

EOS_GAMMA: Sets the polytropic equation of state to use for gas. This will be used instantaneously for operator-split hydro even if cooling or complicated heating is used. Overridden if EOS_HELMHOLTZ or other more complicated equation-of-state models are used. If nothing more complicated is set and this is not set as well, then \(\gamma=5/3\) is used. Note this is the local polytropic index for e.g. adiabatic operations, setting this does not mean that entropy will be constant in the code (there can be initial differences, shocks, conduction, cooling, etc). If you wish to force the gas to lie along a constant entropy adiabat, then you need to use the additional EOS_ENFORCE_ADIABAT flag. If you wish to represent an isothermal equation of state, set this to something close to but not exactly equal to unity (e.g. 1.0001), or else certain operations that divide by \(\gamma-1\) will produce a NaN and crash (whereas for finite values of \(\gamma-1\), they behave just like isothermal, as intended).

EOS_HELMHOLTZ: The gas will use the Timmes & Swesty 2000 equation of state (for e.g. stellar or degenerate gas equations of state). The electron fraction and mean atomic weight are taken as input parameters (specified in the IC file or calculated on-the-fly if the appropriate physics is included); the EOS then takes these, the density, and internal energy, and returns the pressure, temperature, and entropy of the system, needed for the hydrodynamic solver. This is a standard Helmholtz EOS approach, and standard parameters of the EOS can be edited in the files in the helmholtz folder. Necessary tables are provided here or in the wiki section of the GitHub site.

EOS_TILLOTSON: The gas particles (Type=0) will use a Tillotson EOS (Tilloson 1962, “Metallic equations of state for hypervelocity impact,” Tech. rep., General Dynamics, San Diego CA, General Atomic Div.). This is designed for solid or liquid bodies and solid/liquid+vapor mixtures, particularly popular for simulating solid bodies and their impacts (e.g. asteroid collisions, moon-impact simulations, comet impacts in water, etc). When this flag is active, all particles of Type=0 have an assigned material composition specified by the parameter “CompositionType” (which should be specified in the initial conditions file, or otherwise hard-coded somehow into the start-up operations). This will invoke pre-tabulated equations of state for different compositions: CompositionType=1,2,3,4,5,6 corresponds to granite,basalt,iron,ice,olivine/dunite,or water, respectively (with parameters compiled in Reinhardt+Stadel 2017,MNRAS,467,4252, Table A1). If CompositionType=0, a custom EOS will be used specified by the parameters in the parameterfile (if no particles have CompositionType=0, then these parameters will be ignored). (Exception: under the EOS_TYPES_DEFAULTGAS_AND_SOLIDS option – automatically enabled by GRAIN_FLUID_PROMOTION – CompositionType=0 instead denotes the standard gas fluid, with solid materials using CompositionType=1..6.) Per-particle CompositionType is read from the initial-conditions file by default and preserved (values outside the valid material range are reset to 0); set IO_COMPOSITIONTYPE_NOT_IN_ICFILE if your IC has no CompositionType block and composition is assigned at runtime. The Lagrangian methods in GIZMO are particularly well-suited to these simulations; for detailed demonstrations and example simulations, see Deng et al., arXiv:1711.04589. Note because of the composition assumptions, this should only be used with fixed-mass methods (MFM or SPH). Users of this module should cite Reinhardt+Stadel for the equations-of-state (unless a new one is used/implemented), and Deng et al. for the numerical implementation.

EOS_ANEOS: The gas particles (Type=0) will use a tabulated ANEOS/SESAME equation of state, read from pre-computed lookup tables in (density, temperature) space. This is designed for planetary science applications such as giant impacts, asteroid collisions, and other problems involving realistic material equations of state with phase transitions. Multiple materials can be loaded simultaneously (up to 7), with each particle assigned a material via the “CompositionType” field in the initial conditions file (0-indexed, so CompositionType=0 uses the first table, CompositionType=1 the second, etc.; under EOS_TYPES_DEFAULTGAS_AND_SOLIDS CompositionType=0 instead denotes the standard gas fluid and the tables start at CompositionType=1). Per-particle CompositionType is read from the IC and preserved (out-of-range values reset to 0); set IO_COMPOSITIONTYPE_NOT_IN_ICFILE if the IC has no CompositionType block. The tables should be in SESAME ASCII format with columns for density, temperature, pressure, specific internal energy, specific entropy, and optionally sound speed and phase flag. The number of materials and file paths for each table are specified at runtime in the parameterfile via AneosNumMaterials and AneosTable0, AneosTable1, etc. Tables in the Stewart group SWIFT format (e.g. M-ANEOS forsterite from Stewart et al. 2019 or iron from Stewart 2020) can be converted using the provided convert_stewart_table.py script in the test suite. The code performs bilinear interpolation in log(density)-log(temperature) space and uses Newton-Raphson iteration (with bisection fallback) to invert the energy-temperature relation at each timestep. Pressure, sound speed, temperature, entropy, specific heat, Gruneisen parameter, and material phase are all computed from the tables. Note that EOS_ANEOS and EOS_HELMHOLTZ are mutually exclusive. This should only be used with fixed-mass methods (MFM or SPH).

EOS_ELASTIC: The “gas” particles (Type=0) will be treated as elastic or plastic or visco-elastic substances (for e.g. solid-body models and collisions). In addition to normal isotropic pressure (obeying whatever equation-of-state is specified), the full anisotropic stress forces are solved. The deviatoric stress tensor is evolved according to Hooke’s law, with a von Mises yield model to account for plasticity. When active, the user must specify the shear modulus and elastic limit in the parameterfile, or specify a material composition for each particle if EOS_TILLOTSON is active (in which case one of the pre-tabulated compositions can be used). Note because of the composition assumptions, this should only be used with fixed-mass methods (MFM or SPH).

PLANET_HEATING: Adds a simple internal heating source for solid planetary bodies: radiogenic decay (e.g. of \(^{26}$Al and $^{60}$Fe) plus a background accretional heating term. Once per timestep (operator-split from the hydro), each active Type-0 cell's specific internal energy is increased by $(Q_{\rm rad}\,e^{-t/\tau_{\rm rad}} + Q_{\rm acc})\,\Delta t\), where \(t\) is the simulation time. The rates are set by the parameters PlanetHeating_RadQ0_cgs (initial radiogenic specific heating rate, erg g${–1}$ s${–1}$), PlanetHeating_RadTau_cgs (radiogenic e-folding time in s; set very large for near-constant heating, or =0 to turn the radiogenic term off) and PlanetHeating_AccQ0_cgs (constant background accretional heating rate, erg g${–1}$ s${–1}$; =0 to disable). Resolved impact heating is already captured by the hydro solver, so \(Q_{\rm acc}\) is meant only for unresolved accretion. Enabling this automatically turns on EOS_TILLOTSON. It can be combined with COOLING, but the heating is applied to every Type–0 cell regardless of material, so in mixed gas+solid runs gas cells are heated as well.

Note that these are not the only available equations-of-state in GIZMO, but just a subset of the available choices. For example, turning on radiative cooling physics can produce arbitrarily complex local temperature-density distributions, with modules like COOL_LOW_TEMPERATURES that include molecular and atomic/ionized gas interpolating between the relevant local adiabatic indices \(\gamma\) depending on the molecular fraction of the gas. EOS_TRUELOVE_PRESSURE adds artificial pressure floors desired on some problems. EOS_GMC_BAROTROPIC enforces an exact barytropic equation-of-state relevant for some idealized problems in star formation. EOS_SUBSTELLAR_ISM adopts a more complex equation of state relevant for mixtures of atomic and molecular. These custom equation-of-state options are described below but included in the sections with the flags relevant for the physics they assume (e.g. star formation).

EOS_SUBSTELLAR_ISM: This allows for the local equation of state polytropic index to vary between 7/5 and 5/3 (and occasionally outside this range) following the detailed fit from Vaidya et al. A&A 580, A110 (2015) for densities around \(n_{H} \sim 10^{7} {\rm cm^{-3}}\) for atomic plus molecular mixtures (relevant in e.g. star formation simulations), which accounts for collisional dissociation at 2e3 K and ionization at 1e4 K, and take the molecular-fraction-weighted average to interpolate between atomic/not self-shielding and molecular/self-shielding. Gamma should technically really come from calculating the species number-weighted specific heats, but since the molecular fraction is itself only approximate in the regime where it matters, this should be OK. See the code and uncomment the noted lines in EOS.c if you want to use the exact version from Vaidya et al. 2015, which rolls the heat of ionization into the EOS. Users of this module should cite Grudic et al., arXiv:2010.11254.

EOS_GMC_BAROTROPIC: This implements the idealized barotropic EOS calibrated to simplified radiation-hydrodynamic simulations Masunaga & Inutsuka 2000; useful for test problems in small-scale star formation such as cloud collapse, jet launching. See Federrath et al. 2014ApJ…790..128F. You can also set this parameter to a numerical value =1 to instead use EOS used in Bate Bonnell & Bromm 2003.

Nuclear Reaction Networks

##-----------------------------------------------------------------------------------------------------
#---------------------------------------- Nuclear Reaction Networks
#NUCLEAR_NETWORK                 # top-level switch: enable live nuclear burning (species mass fractions evolved each timestep)
#NUCLEAR_NETWORK_SOLVER=0        # solver: 0=built-in aprox13 alpha-chain (13 species, no external deps), 1=SkyNet, 2=Torch
#NUCLEAR_NETWORK_NSPECIES=13     # number of isotopes tracked (set automatically for aprox13; user-specified for external solvers)
#NUCLEAR_NETWORK_SCREENING       # enable plasma screening corrections (Chugunov+07/Yakovlev+06) for reaction rates
#NUCLEAR_NETWORK_NEUTRINOS       # enable neutrino emission/absorption bands (reuses RT infrastructure for nu_e, nu_ebar, nu_x transport)
#NUCLEAR_NETWORK_NSE_TABLE       # use tabulated NSE (nuclear statistical equilibrium) lookup for T > T_NSE instead of integrating
##-----------------------------------------------------------------------------------------------------

NUCLEAR_NETWORK: Top-level switch enabling live nuclear burning in the simulation. When active, each gas particle carries an array of isotope mass fractions (stored in Metallicity[] slots beyond the standard metal species) that are evolved each timestep according to thermonuclear reaction rates. The nuclear energy generation rate is coupled to the particle’s internal energy, providing self-consistent nuclear heating. This module is designed for problems such as thermonuclear supernovae (Type Ia), white dwarf mergers, X-ray bursts, neutron star accretion disks, and any scenario where nuclear energy release is dynamically important. The nuclear solver runs as a per-particle embarrassingly-parallel loop, making it efficient on both CPUs and GPUs (it uses the same Kokkos gather-dispatch-scatter pattern as the cooling solver). Requires EOS_HELMHOLTZ for the degenerate equation of state. Users should cite Hopkins et al. (in prep.) for the numerical implementation.

NUCLEAR_NETWORK_SOLVER: Selects the nuclear reaction network backend. =0 (default): the built-in aprox13 alpha-chain network, which tracks 13 species (He4, C12, O16, Ne20, Mg24, Si28, S32, Ar36, Ca40, Ti44, Cr48, Fe52, Ni56) with forward and reverse rates from Timmes (1999). This requires no external libraries and is GPU-callable. =1: the SkyNet network library (Lippuner & Roberts 2017), which supports arbitrary isotope lists including r-process nucleosynthesis with hundreds of species, and includes weak interaction rates and neutrino interactions. SkyNet must be separately installed and linked (see Makefile for SkyNet linking examples). =2: the Torch network (Timmes, Hoffman, & Woosley 2000), another external network supporting large isotope sets. Both external solvers are called per-particle during the nuclear burning substep.

NUCLEAR_NETWORK_NSPECIES: The number of isotopes tracked per particle. For the built-in aprox13 solver, this is automatically set to 13 and should not be changed. For external solvers (SkyNet, Torch), this must match the number of species in the network definition file. The isotope mass fractions are stored in the particle’s Metallicity[] array starting after the standard metal species slots, so NUM_METAL_SPECIES is automatically adjusted to accommodate both the standard metals and the nuclear species.

NUCLEAR_NETWORK_SCREENING: Enables plasma screening corrections for thermonuclear reaction rates, following the formalism of Chugunov, DeWitt, & Yakovlev (2007) and Yakovlev et al. (2006). Screening enhances reaction rates in dense, degenerate plasma by accounting for the electrostatic potential of the surrounding ion distribution. This is important for accurate burning rates in white dwarf interiors and neutron star envelopes. The screening factors are computed per-particle from the local density, temperature, and composition, and multiply the bare nuclear reaction rates before integration.

NUCLEAR_NETWORK_NEUTRINOS: Enables neutrino emission and absorption as separate radiation bands, reusing the existing radiation transport infrastructure. Three neutrino species are tracked: electron neutrinos (\(\nu_e\)), electron anti-neutrinos (\(\bar{\nu}_e\)), and heavy-flavor neutrinos (\(\nu_x\), representing \(\nu_\mu\), \(\bar{\nu}_\mu\), \(\nu_\tau\), \(\bar{\nu}_\tau\) combined). Each species is treated as a frequency bin in the RT framework, with emission rates computed from the nuclear network (beta decays, electron captures, pair annihilation) and absorption via charged-current interactions. This is primarily relevant for core-collapse supernova simulations and neutron star merger remnants where neutrino transport affects the electron fraction and hence the nucleosynthesis yields.

NUCLEAR_NETWORK_NSE_TABLE: At sufficiently high temperatures (\(T \gtrsim 5 \times 10^9\) K), nuclear reactions are fast enough that the composition reaches nuclear statistical equilibrium (NSE) on timescales much shorter than the hydrodynamic timestep. When this flag is enabled, the code uses a pre-computed NSE lookup table (in density-temperature-electron fraction space) instead of integrating the full reaction network at these temperatures. This significantly accelerates simulations of detonations and deflagrations where large volumes of material pass through NSE conditions. The NSE table is generated from the same reaction rate data as the network and ensures thermodynamic consistency.

Magnetic Fields

## -----------------------------------------------------------------------------------------------------
# --------------------------------- Magneto-Hydrodynamics
# ---------------------------------  these modules are public, but if used, the user should also cite the MHD-specific GIZMO methods paper
# ---------------------------------  (Hopkins 2015: 'Accurate, Meshless Methods for Magneto-Hydrodynamics') as well as the standard GIZMO paper
#MAGNETIC                       # top-level switch for MHD, regardless of which Hydro solver is used
#MHD_B_SET_IN_PARAMS            # set initial fields (Bx,By,Bz) in parameter file
#MHD_NON_IDEAL                  # enable non-ideal MHD terms: Ohmic resistivity, Hall effect, and ambipolar diffusion (solved explicitly); Users should cite Hopkins 2017, MNRAS, 466, 3387, in addition to the MHD paper
#MHD_NON_IDEAL_CORRECTIONTERMS  # add anomalous-resistivity and Epstein-like drift/slip cross-section corrections to the non-ideal MHD coefficients computed in eos.cc. Refines Ohmic, Hall, and ambipolar coefficients in regimes where the strict-equilibrium and thermal-velocity approximations break down. Cite Hopkins & Squire 2024, arXiv:2405.06026
#MHD_BATTERY_MECHANISMS=1       # in-situ generation of seed magnetic fields from non-ideal EMFs. Bitfield: bit 0 (=1) electron Biermann battery (Kulsrud 1997); bit 1 (=2) radiative-ionization battery (Harrison 1973 / Durrive & Langer 2015); bit 2 (=4) charged-dust battery in TVA / fluid-limit (Soliman, Hopkins & Squire 2025 Eq. 17, mostly-neutral form, driven by dust-vs-gas radiation-pressure differential acceleration); bit 3 (=8) charged-dust battery with explicit dust current J_d summed from grain super-particles (Soliman+ 2025 Eq. 9 with alpha_O/alpha_H/alpha_A coefficients). Bits combine by addition (e.g. =3 = Biermann + RI). Requires MAGNETIC. Bit 1 requires RT_EVOLVE_FLUX + RT_CHEM_PHOTOION. Bits 2 and 3 require GRAIN_FLUID; bit 3 additionally requires RT_OPACITY_FROM_EXPLICIT_GRAINS. Bits 2 and 3 are mutually exclusive. Cite Soliman, Hopkins & Squire 2025, ApJ, 985, 55, when using the dust bits.
#MHD_CONSTRAINED_GRADIENT=1     # use CG method (in addition to cleaning, optional!) to maintain low divB: set this value to control how aggressive the div-reduction is:
                                # 0=minimal (safest), 1=intermediate (recommended), 2=aggressive (less stable), 3+=very aggressive (less stable+more expensive). [Please cite Hopkins, MNRAS, 2016, 462, 576]
#MHD_MODIFIED_GRADIENT          # use Modified-Gradient (MG) method for machine-precision div(B)=0 via sparse matrix solve. Mutually exclusive with MHD_CONSTRAINED_GRADIENT. cite Tu, Wang, Gao & Tang 2026, arXiv:2603.04077
##-----------------------------------------------------------------------------------------------------

MAGNETIC: Enables magnetohydrodynamics (default is ideal MHD). The code will evolve magnetic fields along with everything else, evolving the induction equation, magnetic forces, etc. Specific sub-options (if you wish to disable different parts of the MHD calculation can be found in allvars.h under the “MAGNETIC” heading). Currently, this is done for SPH following Tricco+Price and for non-SPH methods following Gaburov & Nitadori 2010. Both methods use the hyperbolic-parabolic Dedner 2002 scheme for “divergence cleaning” (maintaining the \(\nabla\cdot{\bf B}=0\) constraint, which is numerically challenging). Snapshots will include the B-field and \(\nabla\cdot{\bf B}\). We always use the units convention \(\mu_{0} = 1\) for MHD. The MHD code works in both “regular” and co-moving integrations.

MHD_B_SET_IN_PARAMS: Set the initial magnetic fields to a constant value in the parameterfile (specifying Bx/By/Bz, in Gauss). If this is enabled the code will search for this in the parameterfile (unless you are restarting from a restart or snapshot file with flags 1 or 2), otherwise, it will assume the fields are given in the initial conditions file (with a Bx/y/z value for every particle).

MHD_NON_IDEAL: Enables non-ideal MHD terms, specifically Ohmic resistivity, ambipolar diffusion, and the Hall effect. These are solved operator-split from the ideal MHD Reimann problem, as anisotropic diffusion terms appearing in the induction equation. Currently the numerical implementation of the induction equation terms is well-tested; the developer (PFH) asks that you cite the methods paper Hopkins 2017, MNRAS, 466, 3387, in addition to the GIZMO MHD methods paper, if you use these. However, the coefficients, which the code attempts to calculate based on the physical state of the gas, require careful calibration and testing for physical problems of interest, and authors should explore these for themselves. Some quite detailed modules are already in-code for problems of e.g. star and planet formation and proto-stellar/planetary disk dynamics as well as molecular cloud dynamics: details are in the notes_nonidealmhd_in_ppds file (here).

MHD_NON_IDEAL_CORRECTIONTERMS: Adds approximate corrections to the non-ideal MHD coefficients (Ohmic resistivity \(\eta_{\rm O}\), Hall coefficient \(\eta_{\rm H}\), ambipolar coefficient \(\eta_{\rm A}\)) computed self-consistently from local gas state when MHD_NON_IDEAL is active. The default coefficients assume thermal velocities and strict charge/momentum-balance equilibria; the corrections here account for (1) anomalous resistivity at very large \(|\nabla {\bf B}|\) where the drift speed of the current-carrying species exceeds the local thermal speed, and (2) Epstein-like drift/slip-dependent cross sections that modify the collision integrals when the relative drift between charged species and neutrals is non-negligible compared to thermal speeds. These can be important in cool dense neutral gas (proto-stellar disks, dense molecular cores) where the default coefficients can otherwise be off by factors of a few. The scalings used here are derived and presented in Hopkins & Squire 2024 (arXiv:2405.06026); please cite this paper in addition to the MHD methods paper if you enable this module.

MHD_BATTERY_MECHANISMS: Enables in-situ generation of seed magnetic fields from non-ideal EMFs that source the magnetic field even when \({\bf B} = 0\) initially. This is a bitfield, where each bit enables a physically distinct battery mechanism that can be combined via addition. Bit 0 (=1) is the standard electron Biermann battery, \({\bf E}_{\rm B} \propto \nabla P_{e} \times \nabla n_{e} / (n_{e}^{2} e)\), the conventional source term in fully-ionized plasmas (Kulsrud 1997). Bit 1 (=2) is the radiative-ionization battery from photoionization momentum deposition, where photons deliver excess momentum to electrons relative to ions (Harrison 1973; Durrive & Langer 2015), implemented per-cell as \(\boldsymbol{\alpha}_{k} = (\sigma_{k,{\rm HI}} n_{\rm HI} + \sigma_{k,{\rm HeI}} n_{\rm HeI} + \sigma_{k,{\rm HeII}} n_{\rm HeII}) / (n_{e}\,e\,c)\) summed over the radiation bands \(k\) and dotted into the local radiation flux \({\bf F}_{{\rm rad},k}\). Bit 2 (=4) is the charged-dust battery in the small-stopping-time (terminal-velocity-approximation, TVA) “mostly-neutral” form from Soliman, Hopkins & Squire 2025 Eq. 17: the dust battery EMF is driven by the dust-vs-gas differential acceleration from radiation pressure (per-band \(\delta {\bf a}_{dg} = \sum_{k} (\kappa_{{\rm dust},k} - \kappa_{{\rm gas},k}) {\bf F}_{{\rm rad},k}/c\), with the IR band’s dust/gas split taken from the existing rt_kappa_adaptive_IR_band helper, photoionizing bands assumed gas-dominated, and other bands assumed dust-dominated), then closed via the cool-plasma mobility weighting \({\bf E}'_{{\rm bat},d} = -(m_{d}\,\delta{\bf a}_{dg}/q_{d}) \mu_{d} \omega_{en} \omega_{pn} / [\mu_{d}\omega_{en}\omega_{pn} + \mu_{e}\omega_{pn}\omega_{dn} + \mu_{p}\omega_{en}\omega_{dn}]\), using the same collision frequencies (Pinto & Galli 2008 e-n and i-n rate coefficients, Epstein dust-neutral coefficient) and Draine & Sutin 1987 grain-charge equilibrium already used by the existing non-ideal MHD coefficient calculation in eos.cc (the SHS25 §2.4 proof shows the existing Wardle \(\alpha_{\rm O,H,A}\) coefficients are equivalent to the SHS25 forms in the \(J_{d} \to 0\) limit). Bit 3 (=8) is the same dust battery in the explicit-current form (SHS25 Eq. 9), where the per-cell dust current \({\bf J}_{d} = -\sum_{\rm grain} q_{d} n_{d} ({\bf v}_{d} - {\bf v}_{g})\) is summed in solids/grain_physics_loop.cc from the live GRAIN_FLUID super-particles (piggybacking on the existing grain \(\to\) gas pass so no duplicate neighbor walk is required) and then dotted through the SHS25 \(\alpha_{\rm O}/\alpha_{\rm H}/\alpha_{\rm A}\) tensor as \({\bf E}'_{{\rm bat},d} = -\alpha_{\rm O}\,{\bf J}_{d} - \alpha_{\rm H}\,({\bf J}_{d}\times\hat{\bf b}) - \alpha_{\rm A}\,({\bf J}_{d}\times\hat{\bf b})\times\hat{\bf b}\) — the explicit form is preferred when grain super-particle dynamics are actually resolved, while the TVA form is recommended for cosmological / CNM regimes where the dust population is sub-resolution. The two dust paths (bits 2 and 3) are mutually exclusive. All bits require MAGNETIC; bit 1 additionally requires RT_EVOLVE_FLUX and RT_CHEM_PHOTOION; bits 2 and 3 require GRAIN_FLUID, and bit 3 additionally requires RT_OPACITY_FROM_EXPLICIT_GRAINS. The battery EMF from all enabled bits is accumulated into cell.E_battery_T2_cell per cell and combined into the divergence-cleaned induction equation in the standard MHD flux loop. Please cite Soliman, Hopkins & Squire 2025, ApJ, 985, 55 (and the underlying Biermann, Harrison, and Durrive-Langer papers as appropriate) for any use of these modules.

MHD_CONSTRAINED_GRADIENT: Activate this to use the hybrid constrained-gradient method of Hopkins 2015 to further reduce div-B errors (beyond the standard Dedner 2002 scheme described above, which is always enabled when you turn on MAGNETIC). This will perform an extra iteration over particles in the gradient step, to iteratively solve for the reconstruction at faces which goes through all particle-averaged central values and minimizes the divergence errors, while also minimizing the least-squares divergence from the default gradient estimator-based reconstruction at the faces. It is slightly more expensive, but nearly as accurate as constrained-transport methods (which preserve div-B to machine accuracy) in most problems. See the methods paper: Hopkins 2015. You can choose how aggressively you wish to control the errors by setting this value: 0=minimal (safest and most stable, but relatively diffusive formulation which does less to control divergences), 1=intermediate (recommended case), 2=aggressive (this will be somewhat more expensive, doing an additional iteration, and uses weaker slope-limiters, which allows more successful divergence control, but can sacrifice numerical stability on certain types of problems). Please cite Hopkins, MNRAS, 2016, 462, 576, if you use these methods, as they were first presented therein.

MHD_MODIFIED_GRADIENT: Activate this to use the Modified-Gradient (MG) method of Tu, Wang, Gao & Tang (2026, arXiv:2603.04077) for machine-precision divergence-free magnetic field reconstruction. Unlike the CG method above, MG formulates the divergence constraint as a symmetric positive-definite sparse linear system and solves it using an algebraic multigrid (AMG) preconditioned conjugate gradient solver (via HYPRE, if available, or Intel MKL PARDISO). This achieves true machine-precision div(B)=0 at the cost of a global sparse solve per (active) timestep. The runtime parameter ActiveFractionForMGSweep (default: apply every step) can be set to a value between 0 and 1 to limit the MG correction to timesteps where at least that fraction of cells are active, with the cheaper CG method used on smaller steps. This flag is mutually exclusive with MHD_CONSTRAINED_GRADIENT. Please cite Tu, Wang, Gao & Tang 2026 if you use this method. When the MG sweep is skipped on a small step (active fraction below ActiveFractionForMGSweep), the previous MG solution is retained and the cheaper MHD_CONSTRAINED_GRADIENT-style local face correction runs on top of the stale MG state, correcting only the small drift accumulated for those cells that have advanced since the last MG solve (rather than starting from scratch). In test problems, this MG+CG combination with adaptive ActiveFractionForMGSweep has been shown to reduce \(|\nabla\cdot{\bf B}|\) by ~4 orders of magnitude compared to CG alone (which itself reduces \(|\nabla\cdot{\bf B}|\) by ~2 orders of magnitude compared to pure Dedner cleaning) — so while not formally achieving machine-precision \(\nabla\cdot{\bf B}=0\) on every step, the adaptive scheme captures most of the divergence-control benefit of the full MG solve at a small fraction of the cost.

Two-Temperature Plasma (Independent Electron Temperature)

##-----------------------------------------------------------------------------------------------------
#--------------------------------------- Two-Temperature Plasma
#TWO_TEMPERATURE_PLASMA=1       # evolve T_e independently from the ion/gas temperature T_i. Bitfield: bit 0 (=1) electron-ion Coulomb (Spitzer) equilibration + radiative cooling/Compton routed to T_e [core]; bit 1 (=2) electron-neutral elastic coupling [reserved, not yet implemented]; bit 2 (=4) route the Spitzer conduction deposition into u_e instead of u_total (requires CONDUCTION). Combine bits with addition. Requires COOLING. Mutually exclusive with CHIMES, EOS_HELMHOLTZ, and COOLING_OPERATOR_SPLIT (v1).
##-----------------------------------------------------------------------------------------------------

TWO_TEMPERATURE_PLASMA: Enables an independently-evolved electron temperature \(T_{e}\) separate from the ion/gas temperature \(T_{i}\), for problems where the electron-ion Coulomb equilibration timescale becomes comparable to or longer than other timescales of interest (e.g. shocks in dilute hot plasma, intracluster medium, supernova-remnant interiors, hot accretion flows). The primary state carried per cell is the specific electron internal energy per unit gas mass, \(u_{e,{\rm cell}} = (3/2) n_{e} k_{\rm B} T_{e} / \rho\); the electron temperature \(T_{e,{\rm cell}}\) is a derived cache. The total internal energy InternalEnergy remains the single conserved hydrodynamic variable – the module only locally re-partitions it into electron and ion sub-energies (sub-energy mode), so the Riemann solver, snapshot consumers, and downstream couplings (including the Biermann battery in MHD_BATTERY_MECHANISMS, the gradient pass, and snapshot I/O) see the same total \(u\) as in the single-temperature case. This is a bitfield: bit 0 (=1, required core) activates electron-ion Coulomb (Spitzer 1962) thermal equilibration with rate \(\alpha_{ei} \propto n_{e} n_{i} \Lambda_{ei} / T_{e}^{3/2}\) and routes the radiative cooling and Compton-heating components of the cooling-function update to \(T_{e}\) alone (rather than the total), so the cooling rate inside the rootfind is physically evaluated as a function of \(T_{e}\) as it should be; bit 1 (=2) is reserved for electron-neutral elastic coupling, not yet implemented (enabling it will hard-fail at compile); bit 2 (=4) routes the Spitzer-Härm thermal conduction deposition (when CONDUCTION is active) into \(u_{e}\) rather than \(u_{\rm total}\), since electron heat conduction in a magnetized plasma is physically an electron-channel energy transport. By default, the fraction of hydro dissipation deposited into electrons at shocks is controlled by the runtime parameter TwoTemp_ShockElectronFraction (default \(f_{e}=0\), the collisionless-shock limit following Vink+15, Ghavamian+13, where ions absorb all dissipation and electrons heat only via Coulomb equilibration; setting \(f_{e}=1\) recovers strong-coupling single-fluid behavior). The initial \(T_{e}/T_{i}\) ratio at startup is controlled by TwoTemp_InitialTeOverTgas (default 1.0 = LTE seed; set \(\neq 1\) to initialize in a non-equilibrium state, e.g. for relaxation regression tests). Requires COOLING. Mutually exclusive in v1 with CHIMES (which evolves \(T_{e}\) internally already), EOS_HELMHOLTZ (which owns \(T\) directly via the tabulated EOS), and COOLING_OPERATOR_SPLIT (the current implementation uses the unsplit cooling rootfind path). Mach-dependent \(f_{e}(\mathcal{M})\) (Ghavamian-style ramp) is deferred to a future refinement.

Conduction & Viscosity

##-----------------------------------------------------------------------------------------------------
#--------------------------------------- Conduction
#------------------------------------------ [Please cite and read the methods paper Hopkins 2017, MNRAS, 466, 3387]
#CONDUCTION                     # Thermal conduction solved *explicitly*: isotropic if MAGNETIC off, otherwise anisotropic
#CONDUCTION_SPITZER             # Spitzer conductivity accounting for saturation: otherwise conduction coefficient is constant  [cite Su et al., 2017, MNRAS, 471, 144, in addition to the conduction methods paper above].  Requires COOLING to calculate local thermal state of gas.
##-----------------------------------------------------------------------------------------------------
##-----------------------------------------------------------------------------------------------------
#--------------------------------------- Viscosity
#------------------------------------------ [Please cite and read the methods paper Hopkins 2017, MNRAS, 466, 3387]
#VISCOSITY                      # Navier-stokes equations solved *explicitly*: isotropic coefficients if MAGNETIC off, otherwise anisotropic
#VISCOSITY_BRAGINSKII           # Braginskii viscosity tensor for ideal MHD [cite Su et al., 2017, MNRAS, 471, 144, in addition to the viscosity methods paper above]. Requires COOLING to calculate local thermal state of gas.
##-----------------------------------------------------------------------------------------------------

CONDUCTION: Turn on physical, isotropic or anisotropic thermal conduction. This enables physical conduction for irregular particle/grid distributions, implemented in an operator-split manner. If MAGNETIC is off, the conduction is assumed to be isotropic; if MAGNETIC is on, the conduction is anisotropic (according to the magnetic field direction as it should be). This is implemented for both SPH (following Cleary+Monaghan 1999) and MFM/MFV modes (derived from scratch for our meshless formalism, but very similar to known implementations in other arbitrary-mesh finite-volume methods). For SPH, be careful: conduction relies on the (noisy and zeroth-order inconsistent) SPH second-derivative operator. So a large kernel is especially useful to minimize the systematic errors. For MFM/MFV methods, the consistent finite-volume formulation is used; this is far more accurate, less noisy, and converges more rapidly, especially for the anisotropic case. In either case, since we solve the conduction equations explicitly, a stronger timestep restriction is necessary (since the equations are parabolic); this is self-consistently calculated. The methods paper for these implementations is Hopkins 2017, MNRAS, 466, 3387; please cite this if these modules are used.

CONDUCTION_SPITZER: Calculate the thermal conductivities of the gas self-consistently following Spitzer, accounting for saturation (when the gradient scale lengths approach the electron mean free paths). This module requires COOLING is active or some other chemical module is active, because the coefficients depend on thermodynamic variables such as temperature, ionization state, etc (otherwise fully-ionized monatomic primordial gas assumed). If this is off, the coefficients are simply constants set by hand. In addition to the conduction methods paper, cite Su et al., 2017, MNRAS, 471, 144, where these calculations were presented and the numerical calculation of said terms was developed.

VISCOSITY: Turn on physical, isotropic or anisotropic viscosity. By default this enables Navier-Stokes viscosity, with shear and bulk viscosities set in the parameterfile. The operator-splitting is similar to that for conduction. Again this is implemented for both SPH and MFM/MFV modes; but the MFM/MFV mode is much more accurate and highly recommended. If you are going to use SPH, be sure to use a very large kernel. Timesteps are again appropriately limited for solving the physical equations here. As with conduction, in the pure-hydro case, isotropic viscosity is assumed. If MAGNETIC is on, then the physically correct anisotropic tensor viscosity is used. For material simulations, one can simulate visco-elastic materials by setting the appropriate shear and bulk viscosity modulus, and these can be made to depend on local material properties by editing the lines in gradients.c where the particle-carried local coefficients CellP[i].Eta_ShearViscosity and CellP[i].Zeta_BulkViscosity are set (if EOS_TILLOTSON is set, one can assign different properties to different materials specified in the CompositionType flag). The methods paper for these implementations is Hopkins 2017, MNRAS, 466, 3387; please cite this if these modules are used.

VISCOSITY_BRAGINSKII: Calculates the leading-order coefficients for viscosity in ideal MHD for an ionized plasma following Braginskii. This module requires COOLING is active or some other chemical module is active, because the coefficients depend on thermodynamic variables such as temperature, ionization state, etc (otherwise fully-ionized monatomic primordial gas assumed). If this is off, the viscosity coefficients are simply set by hand. In addition to the conduction/viscosity methods paper, cite Su et al., 2017, MNRAS, 471, 144, where these calculations were presented and the numerical calculation of said terms was developed.

Passive Scalars, Metals, & Sub-Grid Turbulent Mixing

The fluids in code can carry arbitrary mixes of scalar fields. These control some of the most-often used. Metals is really a much more generic field, used for tracing different heavy element species, pure passive-scalar tracer fields in many of the optional galaxy and star formation models in the code, dust types, and more. But we also here control different solver options for how those tracers and fields can diffuse or mix, in different ways. 

##-----------------------------------------------------------------------------------------------------
#--------------------------------------- Smagorinsky Turbulent Eddy Diffusion Model
#---------------------------------------- Users of these modules should cite Hopkins et al. 2017 (arXiv:1702.06148) and Colbrook et al. (arXiv:1610.06590)
#METALS                         # enable metallicities (with multiple species optional) for gas and stars [must be included in ICs or injected via dynamical feedback; needed for some routines]
#TURB_DIFF_METALS               # turbulent diffusion of metals (passive scalars); requires METALS
#TURB_DIFF_ENERGY               # turbulent diffusion of internal energy (conduction with effective turbulent coefficients)
#TURB_DIFF_VELOCITY             # turbulent diffusion of momentum (viscosity with effective turbulent coefficients)
#TURB_DIFF_DYNAMIC              # replace Smagorinsky-style eddy diffusion with the 'dynamic localized Smagorinsky' model from Rennehan et al. (arXiv:1807.11509 and 2104.07673): cite those papers for all methods. more accurate but more complex and expensive.
##-----------------------------------------------------------------------------------------------------

METALS: Enables tracking metallicities/passive scalars associated with gas and stars. Enrichment will be handled by the different specific physics modules below (e.g. some of the star formation modules attempt to inject metals on-the-fly), or can be added to the initial conditions (or specified in the parameterfile, if you wish to initialize the simulation to a uniform metallicity and set of abundance ratios). This specific flag does not require GALSF be enabled; if you turn it on without GALSF metals are treated as fixed by the ICs or passive scalars which can diffuse if e.g. models for turbulent metal diffusion are enabled, and can act in the metal-line cooling routines (they just do not have a dynamical mechanism for enrichment). Citation is not required for the specific passive-scalar routines, but the relevant cooling and diffusion routines do ask for appropriate citations.

TURB_DIFF_METALS/TURB_DIFF_ENERGY/TURB_DIFF_VELOCITY: Enables turbulent diffusion following the Smagorinsky (1963) eddy mixing model. This model assumes that for a scalar A, the diffusion time derivative follows \(\nabla\cdot(K\nabla A)\), where the diffusion coefficient \(K = C\,|S|\,h^2\) where h is the interparticle spacing, C is a dimensionless constant set in the parameterfile, and |S| is the norm of the trace-free shear tensor. Essentially this assumes that the grid-scale velocity gradients are unstable and become a cascade, mixing scalars on the eddy turnover time. TURB_DIFF_ENERGY enables diffusion of thermal energy, TURB_DIFF_VELOCITY of momentum (essentially a true ‘turbulent viscosity’), and TURB_DIFF_METALS of metals (treated as passive scalars, requires METALS enabled). All follow the same scaling (required for consistency). The SPH implementation of this is similar to that in Shen et al. 2009, with some numerical tweaks to improve accuracy, conservation, and integration stability; the MFM/MFV implementation follows our more accurate modeling of diffusion terms as described above for conduction and viscosity, with limiters based on a hybrid MFV-solution to prevent excessive diffusivity in many circumstances. Users of these modules should cite Hopkins et al. 2017 (arXiv:1702.06148) and Colbrook et al. (arXiv:1610.06590); a detailed description of the numerical methodology is given in the arXiv:1702.06148 paper, Appendix F3, or in Rennehan et al. (arXiv:1807.11509).

TURB_DIFF_DYNAMIC: This replaces the default code method for calculating the diffusion with the more sophisticated localized dynamical Smagorinsky model developed and tested in Rennehan et al. (arXiv:1807.11509 and 2104.07673). Please cite those papers if you use this for any applications. This requires more computational and memory expense as it performs multiple passes on the velocity field, smoothing it on various scales, to construct more accurate and robust estimators of the true local turbulent diffusivity (as compared to terms coming from shear or resolution-scale noise). If it is important to track turbulent diffusion but the standard code Smagorinsky method is too diffusive, this may yield dramatic improvement. However it is less trivially cross-compatible with all other models, so some care is needed. If this is on, the user must specify a few additional parameters in the params file: (1) TurbDynamicDiffFac sets the ratio of the higher-level filter size to the hydrodynamic kernel size (the smallest filter size): this is the ratio \(\hat{h}/\bar{h}\) in Rennehan et al., and following that paper, optimal behavior appears to result from setting this to \(=2\). (2) TurbDynamicDiffSmoothing sets the smoothing parameter \(\epsilon\) used in weighting the contribution of the local resolution element versus its neighbors in constructing smoothed field values. Only values \(0<\epsilon<1\) are physical, and really only values \(0.7 < \epsilon < 1\) give effective smoothing. Following Rennehan et al. a good compromise default is \(=0.8\). (3) TurbDynamicDiffMax sets the maximum dimensionless value of \(C_{\rm dyn}\) or \(C_{s}^{2}\) (the dimensionless coefficient in the diffusivity, which appears linearly in front of the diffusion rate. You can set this to some very large value to simply allow the code to do whatever it wants, but extremely large values resulting sometimes from numerical error can produce instability. A default value is \(=1\), which should be rarely if ever reached. If you want to enforce lower diffusivity, however, this can be correspondingly lower. Note also that enabling OUTPUT_TURB_DIFF_DYNAMIC_ERROR allows you to record and save to snapshots the error terms tracked in that paper. The multi-iteration form of the dynamic model is disabled: the number of extra filtering iterations is fixed at zero in the code, and a TurbDynamicDiffIterations line in the parameterfile is ignored (the parser reports it as an unused tag).

Dust-Gas Mixtures

In the last few years there have been many simulations by different groups using GIZMO to simulate the dynamics of dust grains in different media. The code can follow the dynamics of arbitrary populations of grains with arbitrary spectra of sizes, charges, chemical composition, including all the major known collisional drag terms, Coulomb drag and other charge effects, Lorentz forces (interactions between grains and magnetic fields), the forces from grains on gas and vice-versa, creation and destruction of grains, grain-grain collisions, dynamical interaction of grains with explicitly-evolved radiation fields, the role of grains determining those fields (i.e. grain re-emission, dust temperatures, and more). Explicit grain dynamics means following individual grain orbits and trajectories, sampled in the same kind of particle-in-cell manner as a standard PIC or N-body gravity code, which is necessary to represent certain kinds of grain distribution functions (as opposed to treating grains as a single fluid, which cannot represent many physical situations because it assumes grains are strongly collisional and will give unphysical answers in many situations). Following grains this way is necessary to capture many of the most interesting grain behaviors, like the streaming instability, resonant drag instabilities, correct grain-grain collision and bouncing/shattering behaviors, and more. Grain self-gravity, we should note, is trivial to add – all you need to do is turn on the standard self-gravity modules, using whatever masses you’ve assigned to the grains.

Examples of different parameterfile choices to set up different grain runs are in the example parameterfile and parameterfile section below.

## ----------------------------------------------------------------------------------------------------
# --------------------------------------- Aerodynamic Particles
# ----------------------------- (This is developed by P. Hopkins, who requests that you inform him of planned projects with these modules
# ------------------------------  because he is supervising several students using them as well, and there are some components still in active development.
# ------------------------------  Users should cite: Hopkins & Lee 2016, MNRAS, 456, 4174, and Lee, Hopkins, & Squire 2017, MNRAS, 469, 3532, for the numerical methods
#GRAIN_FLUID                    # aerodynamically-coupled grains (particle type 3 are grains); default is Epstein drag
#GRAIN_EPSTEIN_STOKES=1         # uses the cross section for molecular hydrogen (times this number) to calculate Epstein-Stokes drag; need to set GrainType=1 (will use calculate which applies and use appropriate value); if used with GRAIN_LORENTZFORCE and GrainType=2, will also compute Coulomb drag
#GRAIN_BACKREACTION             # account for momentum of grains pushing back on gas (from drag terms); users should cite Moseley et al., 2018, arXiv:1810.08214.
#GRAIN_LORENTZFORCE             # charged grains feel Lorentz forces (requires MAGNETIC); if used with GRAIN_EPSTEIN_STOKES flag, will also compute Coulomb drag. Need to set GrainType=2.
#GRAIN_COLLISIONS               # model collisions between grains (super-particles; so this is stochastic). Hard-sphere scattering across pi*(R_i+R_j)^2, weighted by how many grains each super-particle represents (see the scattering-rate equations in the DM_SIDM section); options for inelastic or velocity-dependent terms. Approved users please cite papers above and Rocha et al., MNRAS 2013, 430, 81
#GRAIN_EVOLUTION=(1+2+4+8+16+32+64) # per-super-particle grain size/composition evolution (bitflag)
#GRAIN_FLUID_PROMOTION          # promote massive grain super-particles to Type-0 solid bodies
#GRAIN_RDI_TESTPROBLEM          # top-level flag to enable a variety of test problem behaviors, customized for the idealized studies of dust dynamics in Moseley et al 2019MNRAS.489..325M, Seligman et al 2019MNRAS.485.3991S, Steinwandel et al arXiv:2111.09335, Ji et al arXiv:2112.00752, Hopkins et al 2020MNRAS.496.2123H and arXiv:2107.04608, Squire et al 2022MNRAS.510..110S. Cite these if used.
##-----------------------------------------------------------------------------------------------------

GRAIN_FLUID: Enables the treatment of grain ‘super-particles’ (which will be assigned particle type=3), which couple to the gas via aerodynamic forces. The grain drag equations are solved semi-implicitly for a neutral gas. Arbitrary size spectra can be used, as the grains are ‘super-particles’ so they sample the population in a Monte-Carlo fashion. The implementation of the drag equations here explicitly allows for compressible (non-constant density) gas, and supersonic or sub-sonic gas flows and relative grain-gas motion; this is especially important in GMC-like environments. Parameters of the grain population are specific in the parameterfile. By default, grains are assumed to be in the Epstein regime (size much smaller than the gas mean free path); this makes their dynamics independent of the absolute value of various fluid quantities. These modules are made free to use for users of the development code, but as the developer of the modules, PFH asks that you please inform him of planned projects using the modules – this is because there are several undergraduate and graduate students working with him on projects involving these physics, so we can hopefully avoid any project “collisions.” Users of this module (or any of the GRAIN modules) should cite the methods papers for the basic integration scheme: Hopkins & Lee 2016, MNRAS, 456, 4174, and Moseley, Squire, and Hopkins 2019, MNRAS, 489, 325. Both papers contains details and various rigorous tests of the integration scheme (showing it reproduces linear tests, converges, does not produce spurious clumping): for example, although there are some (much simpler) tracer-particle integration schemes which produce large integration errors between the dust and gas, it is shown there that this scheme produces only about ~0.05 dex of artificial numerical scatter in the separation between dust and gas under “worst case” conditions.

GRAIN_EPSTEIN_STOKES: Uses the physical units of the code (and specified grain parameters) to calculate whether or not grains are actually in the Epstein (size smaller than gas mean free path) or Stokes regime, and modifies the aerodynamic equations appropriately. This is important for very large grains or dense, hot systems, and for terrestrial systems. To determine the gas mean-free path, the cross-section for molecular hydrogen is assumed; however by setting GRAIN_EPSTEIN_STOKES to a value not equal to unity, this cross section is multiplied by that value. This is useful to represent different atmospheric compositions, for example. If this is set and GRAIN_LORENTZFORCE is also active, it will also compute the Coulomb forces on the grains (because it can calculate grain charges); these will use a simple approximation for the ionization state of the gas (needed for the Coulomb drag term) based on its temperature, unless COOLING is also active in which case this is done self-consistently. Grain particles with GrainType=1 are recognized as Epstein/Stokes particles.

GRAIN_BACKREACTION: Explicitly account for the momentum deposited into the grain population (modify the gas momentum appropriately). This requires specifying the absolute dust-to-gas ratio and other physical parameters – that is done in the initial conditions (when the actual masses of dust ‘super-particles’ is assigned), although it can easily be done in the init.c file as well (hard-coding some dust-to-gas ratio). In either case, the dust super-particle masses define the total mass of dust, which in turn determines the strength of its force on the gas (if this is dis-abled, the total mass in dust in the system has no dynamical effect). This module is now public: users should cite Moseley et al., 2018, arXiv:1810.08214.

GRAIN_LORENTZFORCE: Explicitly account for the Lorentz forces on grains (integrated efficiently with a semi-implicit Boris integrator scheme that can accurately preserve gyro orbits, even with numerical discretization errors). The grain charge is calculated self-consistently from grain sizes and densities and ambient gas conditions, assuming the grains obey local charge equilibrium (following Draine and Sutin 1987). Requires MAGNETIC be active. With GRAIN_EPSTEIN_STOKES turned on, this will also compute the electro-static (Coulomb) drag on the grains. See Lee, Hopkins, & Squire 2017, MNRAS, 469, 3532, for details and examples. Grain particles with GrainType=2 are recognized as charged particles (feel both Epstein/Stokes drag and Lorentz force).

GRAIN_COLLISIONS: Enables grain-grain collisions. This uses the same framework as the SIDM modules, in fact, to model collisons/scattering/destruction/sticking of mostly-collisionless species represented via super-particles. If this is on, then neighboring grain super-particles calculate the ensemble number of grain-grain collisions as they move through one anothers interaction kernel radii (calculated in an SPH-like manner assuming the grain distribution is volume-filling, with neighbor number set by the AGS_DesNumNgb parameter). Since the trajectories of individual super-particles are integrated as if they were a single grain, we cannot follow all possible post-collision trajectories at once, but instead the super-particle can then be discretely kicked stochastically with probability equal to the ratio of the true number of collisions relative to the number of grains (represented by the super-particle) in the timestep. By default, the code assumes (by default) simple elastic hard-sphere scattering between grains of the specified size and density, but includes parameters to arbitrarily change the normalization of the rates, to make the interactions dissipative or energetic (inelastic), and to include a velocity-dependence of the grain-grain cross-sections (appropriate if e.g. Coulomb scattering dominates), in the run-time parameters. More detailed models for what happens to grains in collisions (e.g. shattering) can easily be implemented, using the modular functions return_grain_cross_section_per_unit_mass and prob_of_grain_interaction (which determine the probability of grain-grain interactions, as they say) and calculate_interact_kick (which determines the result of an interaction when it occurs), all defined in the grain_physics.c file – users are encouraged to modify these appropriate for the physics at hand. Users should cite Rocha et al., MNRAS 2013, 430, 81, in addition to the other grain methods papers above – this paper describes the numerical methods (for dark matter there, but the fundamental method is the same), accounting for subtleties like integration through the kernel, etc, which are all handed for you as the user.

GRAIN_EVOLUTION: Evolves the size and composition of individual grain super-particles (GRAIN_FLUID, which is required). It is a per-super-particle, stochastic translation of the bin-based grain-size evolution physics of the ISM dust-chemistry module, and is mutually exclusive with GALSF_ISMDUSTCHEM_GRAINSIZEEVO. Each super-particle is monodisperse: its mass, Grain_Size and a six-species composition (silicate, carbon, iron, and H$_2$O, CO and CO$_2$ ices) are its whole state, and the number of physical grains it represents follows from mass and size. The value is a bitflag selecting the processes:
- 1 = coagulation: below the Dominik & Tielens (1997)/Hirashita & Chen (2023) sticking velocity, scaled by GrainEvolution_StickingCoeff, one super-particle absorbs the other.
- 2 = fragmentation: between that velocity and the species shattering velocity, both grains lose up to ~10% of their size-equivalent mass.
- 4 = shattering: above the shattering velocity, larger size reductions.
- 8 = thermal sputtering: Nozawa et al. (2006) rates, scaled by GrainEvolution_ThermalSputteringScaling.
- 16 = non-thermal sputtering, driven by the grain’s drift through the gas.
- 32 = condensation of volatile ices onto grain mantles.
- 64 = sublimation of those ices.

Bits 1, 2 and 4 act in the GRAIN_COLLISIONS pair kernel and require it. Bits 8 and 16 are local operators applied in the drag step. Bits 32 and 64 exchange mass with a set of gas-phase volatile species carried by the gas cells, deposit or remove the latent heat in the gas, and require GRAIN_BACKREACTION. The microphysical constants are shared with the ISM dust-chemistry module. Run-time parameters: GrainEvolution_StickingCoeff, GrainEvolution_VelThreshFrag, GrainEvolution_VelThreshShat and GrainEvolution_ThermalSputteringScaling. Developed by PFH and still in development: please contact the developer before using it.

GRAIN_FLUID_PROMOTION: Converts (‘promotes’) grain super-particles into Type–0 solid bodies once they have grown enough, so that the growth of grains into planetesimal-like bodies can continue with a solid-body equation of state. Requires GRAIN_FLUID and is intended to be used with grain growth (GRAIN_EVOLUTION). Each timestep, an active grain super-particle is promoted if its mass exceeds GrainPromotion_MassThresh_cgs (in grams). If GrainPromotion_DustGasRatioThresh > 0, its local grain-to-gas density ratio (estimated from the grain kernel) must also exceed that value; =0 disables this test. The promoted element becomes a Type–0 cell with a Tillotson material set by its dominant composition species when GRAIN_EVOLUTION is on (silicate to olivine, carbon to basalt, iron to iron, ices to ice; olivine otherwise). It starts at the material’s reference density, with internal energy 1% of the material’s \(u_{s}\) parameter and zero stress. Enabling this automatically turns on EOS_TILLOTSON and EOS_TYPES_DEFAULTGAS_AND_SOLIDS, so that composition-type–0 cells remain ordinary gas. It also turns on IO_COMPOSITIONTYPE_NOT_IN_ICFILE: every initial Type–0 cell is treated as gas and per-cell composition types are not read from the ICs, while restarts from snapshots do read them, so promoted solids survive restarts. ICs containing pre-existing solid bodies are therefore not supported in this mode. The number of promotions is limited by the gas-cell memory allocation (a message is printed if the limit is reached). Developed by PFH and still in development: please contact the developer before using it.

GRAIN_RDI_TESTPROBLEM: This is a top-level flag to enable a variety of test problem behaviors, customized for the idealized studies of dust dynamics in papers like Moseley et al 2019MNRAS.489..325M, Seligman et al 2019MNRAS.485.3991S, Steinwandel et al arXiv:2111.09335, Ji et al arXiv:2112.00752, Hopkins et al 2020MNRAS.496.2123H and arXiv:2107.04608, Squire et al 2022MNRAS.510..110S, Fuentes et al. 2022, Soliman et al. 2022, and others. This in turn enables a wide range of custom behaviors. For example, it enables the following parameterfile parameters: (1) Grain_Charge_Parameter: the dimensionless grain charge parameter, \(\tilde{\phi}_{\rm max}\), from Lee et al. 2017MNRAS.469.3532L and Seligman et al 2019MNRAS.485.3991S, defined for the largest grains; (2) Dust_to_Gas_Mass_Ratio the mean dust to gas mass ratio of the box (grain super-particle masses will be renormalized assuming their initial total mass in the box is unity, to give this value - don’t use this unless your ICs are actually set up appropriately); (3) Vertical_Gravity_Strength: the value of the externally imposed, constant analytic gravitational force, in code units; (4) Vertical_Grain_Accel: the value of the acceleration from external forces (e.g. radiation) on dust grains (in code units), normalized to the largest grains (for simulations with live radiation-dust-magnetohydrodynamics, this normalizes the incident radiation flux to the parameter \(a_{0}\) as described in Hopkins et al. arXiv:2107.04608); (5) Vertical_Grain_Accel_Angle: an arbitrary angle in degrees between the direction of the grain acceleration and gravity (defining e.g. anti-parallel for this equal to zero); (6) Pressure_Gradient_Accel: a parameter defining the acceleration on gas from the background pressure gradient used specifically for radially-stratified (drifting) shearing-box simulations like those in Fuentes et al. 2022 (not used if BOX_SHEARING is not set); (7) Grain_Q_at_MaxGrainSize: value of the dimensionless grain extinction efficiency \(Q\) (cross section relative to geometric, at the wavelength of interest), for the largest grains, if RT_OPACITY_FROM_EXPLICIT_GRAINS is set; (8) Grain_Internal_Density: value of the internal material density of grains in cgs (will be converted to code units inside the code; of course if the code unit parameters for cgs are unity, then this is equivalently in code units); (9) Grain_Size_Min: minimum size of grains in cgs (the grain sizes and internal properties will be rescaled, probabilistically sampling grain sizes so there are equal numbers of grains per logarithmic size interval); (10) Grain_Size_Max: maximum size of grains in cgs (for a single grain size, set Grain_Size_Min=Grain_Size_Max); (11) Grain_Size_Spectrum_Powerlaw: spectral slope of the grain size distribution, defined as total mass per logarithmic interval in grain size, such that the classic MRN size distribution (\(dN/da \propto a^{-3.5}\)) corresponds to Grain_Size_Spectrum_Powerlaw=0.5; (12) Grain_Absorbed_vs_Total_Extinction: fraction of incident radiation absorbed (1-albedo), assumed for grains, if RT_OPACITY_FROM_EXPLICIT_GRAINS and RT_GENERIC_USER_FREQ are set. If this module is used, please cite the papers above, especially Lee et al 2017MNRAS.469.3532L and Moseley et al 2019MNRAS.489..325M (for any use of these modules), Seligman et al 2019MNRAS.485.3991S (if any magnetic/charge terms are used), Squire et al 2022MNRAS.510..110S (if full size spectra are evolved), Steinwandel et al arXiv:2111.09335 (for stratified boxes), Fuentes et al. 2022 (for shearing boxes), Ji et al arXiv:2112.00752 (for simulations which also include cosmic rays),

There are also a couple of sub-flags which are only used if the GRAIN_RDI_TESTPROBLEM flag is enabled, but vary sub-options within this module. These include:

## ----------------------------------------------------------------------------------------------------
#GRAIN_RDI_TESTPROBLEM_ACCEL_DEPENDS_ON_SIZE    # Make the idealized external grain acceleration grain size-dependent; equivalent to assuming an absorption efficiency Q=1. Cite GRAIN_RDI_TESTPROBLEM papers.
#GRAIN_RDI_TESTPROBLEM_LIVE_RADIATION_INJECTION # Enables idealized radiation injection by a source population designed to set up outflow test problems with live radiation-hydrodynamics as in Hopkins et al., arXiv:2107.04608. Cite that paper if this module is used.
#IO_DUST_NOT_IN_ICFILE              # special flag needed if restarting from a snapshot (flag=2) with no dust. Will set dust species via params arguments and assume an MRN size distribution, if GALSF_ISMDUSTCHEM_MODEL is active 
##-----------------------------------------------------------------------------------------------------

GRAIN_RDI_TESTPROBLEM_ACCEL_DEPENDS_ON_SIZE: Turning this on assumes the extinction efficiency \(Q\) equal to a constant, so that the opacities are just some multiple of geometric, appropriate for grains which are large compared to the incident wavelength (\(\lambda \ll a\)). As a result, the assumed radiative acceleration in e.g. simulations with a uniform radiation background will scale as \(\propto a^{-1}\) (which in turn means that the steady-state drift velocities will be grain-size-independent). By default (with this term off), it is assumed that the relative acceleration between dust and gas (driving the drift) is grain-size independent: that is appropriate either in the regime where the acceleration owes to radiation on grains in the long-wavelength limit (\(\lambda \gg a\)), or for regimes where the relative acceleration is driven by radiation on the gas (e.g. line-driving or ionizing radiation, where the gas-phase optical depth is dominant over the dust-phase optical depth), or some other grainsize-independent difference in forces (if e.g. gravity drives the drift, or the gas is stratified so there are smooth large-scale pressure forces in the gas). If this is used, cite the same papers as the GRAIN_RDI_TESTPROBLEM papers described above.

GRAIN_RDI_TESTPROBLEM_LIVE_RADIATION_INJECTION: Turning this on enables idealized radiation injection by a source population designed to set up outflow test problems with live radiation-hydrodynamics as in Hopkins et al., arXiv:2107.04608, Soliman et al 2022. Cite both of these papers if this is used. This is very custom-designed to work with a very specific initial configuration with a specific setup of boundary ghost cells at the lower boundary of a semi-periodic box with various assumptions about the units and other boundary conditions, so we caution against using it for anything other than the very specific setups described therein.

IO_DUST_NOT_IN_ICFILE: Needed when restarting from a snapshot (flag=2) which contains no dust fields, with GALSF_ISMDUSTCHEM_MODEL active. The dust species are then set from the parameterfile arguments and an MRN size distribution is assumed, instead of being read from the file.

Note that the if explicit radiation-hydrodynamics is evolved in the code, you can turn on the code option RT_OPACITY_FROM_EXPLICIT_GRAINS described below (in the RHD section of this User Guide) to enable fully coupled dynamic radiation-dust-magnetohydrodynamics (RDMHD), where the explicit radiation fields interact directly with the dust grains (appropriate when the grains dominate the opacity), and through those dust grains with the gas, as opposed to approximating their interaction as with the gas with some opacity.

Multi-Fluid (Lagrangian-Partition) Framework

GIZMO supports a generic multi-fluid framework in which Type==0 (gas) particles are partitioned into Lagrangian sub-fluids by an integer tag P[].FluidType. Each fluid evolves its own hydrodynamics inside the existing hydro corridor — same density / gradient / pressure-force machinery — but pair operators skip cross-FluidType pairs, so different fluids do not directly exchange mass/momentum/energy through the hydro scheme. Cross-fluid coupling (drag, charge exchange, radiation feedback, etc.) is added by physics-specific sub-modules that explicitly walk cross-FluidType pairs. The base framework is designed primarily as a scaffold for users adding their own new physics: a clean partition of Type=0 into multiple co-evolved fluids, with all the structural plumbing (IC reader, snapshot writer, restart, domain exchange, ghost packing, merge/split inheritance) handled, so the user only writes their cross-fluid coupling. Several reference sub-modules are included to demonstrate the pattern.

## ----------------------------------------------------------------------------------------------------
# --------------------------------------- Multi-Fluid Framework (Lagrangian-partition multi-fluid)
#HYDRO_MULTIFLUID               # base scaffold: Type=0 particles carry a FluidType ID; hydro pair operators skip cross-FluidType pairs (intra-fluid hydro only). Force-implies EOS_GENERAL.
#HYDRO_MULTIFLUID_DUST_DRAG     # cross-fluid drag coupling for Type=0+FluidType=FLUID_DUST_GRAIN dust-as-fluid against Type=0+FluidType=FLUID_DEFAULT gas; reuses the GRAIN_FLUID drag/backreaction kernels. Mutually exclusive with GRAIN_FLUID / HYDRO_MULTIFLUID_IONNEUTRAL.
#HYDRO_MULTIFLUID_IONNEUTRAL    # two-fluid ion-neutral ambipolar drag: Type=0+FluidType=FLUID_ION ions coupled to Type=0+FluidType=FLUID_DEFAULT neutrals via the Draine ambipolar form. Mutually exclusive with GRAIN_FLUID / HYDRO_MULTIFLUID_DUST_DRAG.
#HYDRO_MULTIFLUID_DM            # dark-fluid placeholder: Type=0+FluidType=FLUID_DM particles get a dedicated trivial adiabatic EOS and dedicated do_dark_cooling_for_particle hook. Gas->star/sink promotion redirected to inert Type=3. Baryonic feedback kernels skip FLUID_DM neighbors; RT radiation pressure on FLUID_DM gas zeroed.
#HYDRO_MULTIFLUID_DM_COOLING    # opt-in reference implementation of atomic+molecular ADM cooling, distilled from Roy et al. (cite below). Requires HYDRO_MULTIFLUID_DM. Adds four runtime parameters: ADM_FineStructure, ADM_ProtonMass, ADM_ElectronMass, ADM_MolecularFraction.
##-----------------------------------------------------------------------------------------------------

HYDRO_MULTIFLUID: The base scaffold. Adds an unsigned char FluidType field to P[] (stored in existing struct padding, so no per-particle storage cost on most Configs), and an enum of canonical fluid IDs (FLUID_DEFAULT=0, FLUID_DUST_GRAIN=1, FLUID_ION=2, FLUID_NEUTRAL=3, FLUID_DM=4, with values 5..255 reserved). The HDF5 IC reader looks for a per-particle FluidType dataset on PartType0 and reads it as an integer (defaults to 0 when absent), and snapshots write the field on output. The hydro corridor (density, gradient, cell-corrections, hydro-force loops) adds a top-of-pair-body skip if (!same_lagrangian_fluid_id(ft_i, ft_j)) return; so different fluids do not directly mix hydro state. Cross-fluid mergers in merge_split.cc are hard-refused. When this flag is on but no sub-module (DUST_DRAG / IONNEUTRAL / DM) is enabled, the framework is a no-op: the FluidType field is carried passively but the corridor skip prevents inter-fluid hydro interaction, so users can build their own coupling physics by adding cross-fluid pair body inserts in the relevant loops. The flag force-implies EOS_GENERAL so per-cell EOS state (Pressure, SoundSpeed, Temperature) is saved per-cell rather than re-computed each read — this localizes per-fluid EOS variation to one site (the EOS compute path), which is required for any sub-module that needs a fluid-specific equation of state. Citation: this scaffold is general and lightweight (no specific publication is required for the base flag itself); sub-modules below have their own citations.

HYDRO_MULTIFLUID_DUST_DRAG: Cross-fluid drag coupling between a Type=0 dust-as-fluid (tagged FluidType=FLUID_DUST_GRAIN) and a Type=0 ambient gas (tagged FluidType=FLUID_DEFAULT). Internally, this reuses the existing GRAIN_FLUID aerodynamic drag and backreaction kernels (grain_drag_kernel, grain_backrx_pair_kernel) but widens their source-side eligibility predicate (IS_PARTICLE_DRAGVALID) to also accept Type=0+FLUID_DUST_GRAIN particles. The dust-as-fluid case is physically distinct from the GRAIN_FLUID (Type=3, collisionless dust-as-PIC) case — dust-as-fluid is appropriate when grain stopping times are short compared to the orbital/dynamical timescale (strongly-coupled limit) such that the grain population behaves as a separate fluid rather than as a collisionless PIC sample of grain orbits. The implementation reuses the existing dust drag formula (Epstein + Stokes corrections, with optional Lorentz force under GRAIN_LORENTZFORCE) since these are mechanically identical between Type=3 grain and Type=0 dust-fluid; users implementing more sophisticated dust-fluid microphysics (size-spectrum-resolved fluid mixtures, etc.) can extend the kernel branch directly. Compile-time dependencies: requires HYDRO_MULTIFLUID, GRAIN_FLUID, and GRAIN_BACKREACTION since the kernel/storage are reused; this is an implementation dependency, not a physics dependency — the dust-fluid does not modify or activate Type=3 grain physics. Cite Hopkins & Squire 2018 (MNRAS, 480, 2813) and Soliman et al. 2025 (ApJ, 985, 55) for the underlying dust-MHD framework whose drag/backreaction kernels this re-uses.

HYDRO_MULTIFLUID_IONNEUTRAL: Two-fluid ion-neutral ambipolar drag coupling. Ions are tagged FluidType=FLUID_ION, neutrals FluidType=FLUID_DEFAULT. Ions carry the magnetic field; neutrals carry \(B=0\) in the IC and remain at zero by induction since the intra-fluid corridor skip prevents cross-fluid MHD pair contributions. Cross-fluid drag is computed via the Draine ambipolar form: \(1/t_{\rm stop} = \gamma_{\rm AD}\,\rho_{\rm neutral}\) with \(\gamma_{\rm AD} = \langle\sigma v\rangle_{in} / (m_i + m_n)\). Hardcoded HI/H${}+$ values are used by default: \(\langle\sigma v\rangle_{in} = 1.9\times 10^{-9}\,{\rm cm}^3/{\rm s}\) (Draine 1980, ApJ, 241, 1021) and \(m_i + m_n = 2\,m_p\), giving \(\gamma_{\rm AD} \approx 5.7\times 10^{14}\,{\rm cm}^3\,{\rm g}^{-1}\,{\rm s}^{-1}\) for atomic ISM. Users in other regimes (e.g. HCO${}+$/H$_2$ in molecular clouds, where \(\gamma_{\rm AD} \approx 4\times 10^{13}\,{\rm cm}^3\,{\rm g}^{-1}\,{\rm s}^{-1}\); Mouschovias 1991, ApJ, 373, 169) edit two transparent constants at the top of the ion-neutral drag branch in solids/grain_drag_gpu.cc. Temperature dependence (weak, scaling as \(T^{0.5}\)) is neglected in this default implementation. The existing Epstein drag integrator is retained for the velocity update — it is physically appropriate even for ion-neutral drag, since ambipolar drag becomes nonlinear in an Epstein-like fashion in the supersonic regime (the linear approximation only holds at subsonic relative velocities). Mutually exclusive with GRAIN_FLUID and HYDRO_MULTIFLUID_DUST_DRAG (compile-time enforced by macro-redefinition sentinel). Cite Draine 1980, Hopkins & Squire 2018, and the references in their §2 for the underlying ambipolar drag framework.

HYDRO_MULTIFLUID_DM: Dark-fluid placeholder. Particles tagged FluidType=FLUID_DM get a dedicated trivial adiabatic EOS (\(P = (\gamma-1)\rho u\) with \(\gamma=5/3\)) and a dedicated cooling entry point do_dark_cooling_for_particle. By default the cooling hook is a no-op (dark fluid evolves strictly adiabatically); the opt-in HYDRO_MULTIFLUID_DM_COOLING sub-flag activates a reference atomic+molecular ADM cooling implementation (see below). The framework also handles the cross-coupling of dark fluid with baryonic physics in the standard FIRE/STARFORGE physics stack: (1) gas-to-star and gas-to-sink promotion in galaxy_sf/sfr_eff.cc redirects FLUID_DM particles to inert collisionless Type=3 instead of Type=4 stars or Type=5 sinks, so dark gas that would have formed stars in a standard run becomes a collisionless dark-star-analog tracer; (2) RT radiation pressure on FLUID_DM gas is zeroed in gravity/gravtree.cc (no full RHD coupling — this would have to be added separately for a full mirror-RT model); (3) baryonic stellar feedback kernels (mechanical FB, thermal FB, sink wind/kick, HII heating, local RT) skip FLUID_DM neighbors so dark gas does not feel SNe / wind / radiation feedback from baryonic stars; (4) helper functions ThermalProperties and Get_Gas_Molecular_Mass_Fraction short-circuit for FLUID_DM particles so the full thermochemistry call-stack is bypassed in typical (e.g. FIRE) Configs. The minimum set of edits needed to add a new dark-species sub-module on top of this is small: this single base flag handles the structural plumbing; adding e.g. mirror-RT or DM-DM scattering only requires adding the physics-specific dispatch where relevant. Citation is not specifically required for the base flag; cite the cooling sub-flag references below if the cooling extension is used.

HYDRO_MULTIFLUID_DM_COOLING: Opt-in reference implementation of atomic + molecular cooling for the dark fluid, requiring HYDRO_MULTIFLUID_DM. Implements collisional excitation, collisional ionization, radiative recombination (line + free-bound), and free-free (Bremsstrahlung) cooling for a dark-hydrogen-like fluid with user-supplied fine-structure constant \(\alpha_{\rm ADM}\), proton mass \(m_{p,{\rm ADM}}\), and electron mass \(m_{e,{\rm ADM}}\), plus optional molecular dark-H$_2$ rovib cooling with a user-supplied initial dark-H$_2$ fraction. The implementation lives in sidm/dm_cooling_functions.h (the dark-fluid EOS hook stays in the EOS-only sidm/dm_fluid_functions.h to keep the cooling-table dependency out of broad EOS consumers); the consolidated table struct + builder declaration are in sidm/dm_cooling_tables.h; the __managed__ table instance is owned by cooling/cooling.cc next to CoolTables. The chain uses GIZMO’s in-tree 20-point Gauss-Legendre quadrature for the cross-section rate integrals (no external library dependency). Photoionization is absent by design (no cosmic-microwave-background analog is assumed for the dark fluid; users implementing a dark-sector radiation background should add the term themselves). The cooling chain (implicit bisection solver, ionization-equilibrium balance, ADM-scaled Glover & Abel 2008 / Glover 2015 H$_2$ rovib curves) is largely a cleaned-up consolidation of code originally developed by Roy et al. (see also Ryan et al. 2024 arXiv:2403.05624 and Roy, Hopkins, & Squire in prep) for atomic dark-matter cosmological simulations. This is placeholder reference machinery, not extensively validated against an analytic ADM benchmark in the present code base — users implementing real ADM science should re-validate (a natural first sanity check is to set all ADM_* parameters to their Standard Model values and verify equivalence to the standard COOLING cooling rates). Adds four runtime parameters that must be specified in the parameterfile: ADM_FineStructure (= \(\alpha_{\rm ADM}\); SM value \(\approx 7.2974\times 10^{-3}\)); ADM_ProtonMass (= \(m_{p,{\rm ADM}}\) in g; SM value \(1.6726\times 10^{-24}\)); ADM_ElectronMass (= \(m_{e,{\rm ADM}}\) in g; SM value \(9.1094\times 10^{-28}\)); ADM_MolecularFraction (initial dark-H$_2$ mass fraction, 0..1). Cite Roy et al. and the underlying Glover & Abel 2008 (MNRAS, 388, 1627), Glover 2015 (MNRAS, 451, 2082) for the molecular cooling tables. Users should also note that the base HYDRO_MULTIFLUID_DM scaffold is intentionally minimal — the cooling sub-module is one example of how to extend it, but users with more sophisticated dark-sector microphysics (multi-species, DM-DM scattering, mirror radiation transport, ADM nucleosynthesis, etc.) should treat this as a template to extend, not as a production-ready ADM cooling library.

A general note for users adding their own multifluid sub-modules: the framework intentionally exposes a clean two-loop separation between own-fluid hydro (handled by the existing corridor with the same-FluidType skip) and cross-fluid coupling (handled by sub-module pair bodies inside the existing kernels, dispatched on FluidType predicates). When adding a new sub-module, the typical pattern is (a) reserve a new FluidID enum value in declarations/multifluid_helpers.h; (b) add a new sub-flag to Template_Config.sh (mutually exclusive with other drag-source sub-flags via the existing IS_PARTICLE_DRAGVALID macro-redefinition sentinel if it is a new drag source, orthogonal otherwise); (c) add the cross-fluid pair body insert in the relevant existing kernel(s), gated on the new sub-flag and the appropriate FluidType pair predicate; (d) if the sub-module needs new per-particle state, add it to particle_data gated on the new sub-flag, and extend init.cc / IC reader / snapshot writer / restart / domain exchange / ghost packing as needed (follow the existing Gas_Density / Gas_Velocity / Grain_AccelTimeMin pattern). The framework is deliberately not a generic n-species coupling matrix — each sub-module is its own physics, written to fit the case at hand, with shared infrastructure but separate code paths.

Particle-in-Cell (PIC) and Cosmic Ray-Laden Gas Simulations

GIZMO has been used in many studies to simulate populations of both relativistic and non-relativistic particles in the interstellar and circum/inter-galactic medium, in many different contexts. There are a few different fundamental approaches to doing this, which the code supports. One can simulate individual particle dynamics and trajectories, with large ensembles acting in turn on the gas and each other (i.e. a particle-in-cell or PIC approach), or simulate the evolution of a particle population in the continuum limit – i.e. simulating how its distribution function evolves (this is sometimes referred to as the fluid limit, but strictly speaking its a more general continuum limit that allows for some non-standard and/or collisionless behaviors). And there are methods that evolve these populations as a sub-grid model, where they are not explicitly integrated at all. The relevant options for these are all described below.

MHD-PIC Simulations

These options are used for MHD-PIC simulations of different particles. In the MHD-PIC approximation, one population of particles is represented as a background fluid, which obeys whatever the desired equations of motion would be (e.g. ideal or non-ideal MHD, with or without self-gravity, radiation-MHD, etc). A common simple approximation would be MHD. You specify the behavior of that fluid with the usual flags for whatever physics you want to include. But additional populations are represented on top of this with a population of super-particles (i.e. numerical particles each of which represents many true micro-physical particles, in a Monte Carlo sampling sense), each of which is integrated as if it were a true micro-particle. This is the PIC part. This is most useful when there is a large scale separation in the problem: for example, when some characteristic scale (like the mean free path or gyro radius) of the individual particles in the background fluid (say, the non-relativistic electrons or protons) is very small (unresolved), while it is macroscopically large (resolvable, at least in principle) for the PIC-type particles.

The code can handle arbitrary spectra of different properties of the PIC-type particles, sampled again in Monte Carlo fashion: i.e. different masses/weights, charges, energies/velocities/rigidities/etc.

This is also written to freely handle arbitrary relativistic or non-relativistic velocities. In practice, it has been used to simulate populations of cosmic rays (PIC) moving through background gas (MHD), in Ji, Squire, & Hopkins, arXiv:2112.00752 and Ji & Hopkins, arXiv:2111.14704, dust grains (heavy non-relativistic PIC particles, handled with their own modules above but using the same numerical methods) in an ISM fluid in a huge number of papers (cite e.g. Moseley et al. 2019MNRAS.489..325M), and other species with long mean-free paths (e.g. some special dark matter models). Its up to the user to specify the detailed properties of the desired particles, in the code. But whatever the forces are exerted on the particles, the code calculates the correct equal-and-opposite force back on the fluid, so that ‘back-reaction’ is indeed correctly captured. This means phenomena like cosmic-ray driven winds, resonant drag instabilities, the streaming and gyro-resonant instabilities of both dust and cosmic rays, and other related interactions can all be accurately captured, with various methods papers showing that they produce the correct linear growth rates and non-linear behaviors in idealized test problems.

These are controlled by a couple of simple flags, plus those required for additional physics (noted above), and some user-specified parameters.

## --------------------------------------------------------------------------------------------------
#PIC_MHD                          #- hybrid MHD-PIC simulations for relativistic particles / cosmic rays (particle type=3). need to set 'subtype'. cite Ji, Squire, & Hopkins, arXiv:2112.00752
#PIC_SPEEDOFLIGHT_REDUCTION=1     #- factor to reduce the speed-of-light for mhd-pic simulations (relative to true value of c). requires PIC_MHD. cite Ji & Hopkins, arXiv:2111.14704
#GRAIN_FLUID_AND_PIC_BOTH_DEFINED #- this tells the code that both GRAIN_FLUID (dust) and PIC_MHD (for e.g. cosmic rays or other applications) are simultaneously active. cite Ji, Squire, & Hopkins, arXiv:2112.00752
## --------------------------------------------------------------------------------------------------

PIC_MHD: This is a top-level switch which enables hybrid MHD-PIC simulations for relativistic particles or cosmic rays. The super-particles representing these species will have particle Type=3 by default. However, they will also be labeled with an arbitrary integer sub-type. This sub-type allows you to mix and match populations with arbitrary different properties: for example different masses, charges, etc. The subtype should indicate different species, like electrons versus protons. Subtypes less than or equal to 2 will be assumed to be dust species, which are non-relativistic and experience drag forces in addition to Lorentz forces. Subtypes greater than or equal to 3 are collisionless except for Lorentz forces, by default, though you can explicitly include collisions or catastrophic losses if desired by adding a simple mass loss rate. You can set several subtypes if desired, and note that its actually quite easy to simulate both dust and cosmic rays in the same box, if you want to, as they use the same numerical modules (and often have very similar gyro radii, see e.g. Squire et al., 2021MNRAS.502.2630S). Note that the energies/rigidities are determined self-consistently from the particle individual masses and velocities – the velocities are set as usual in the initial conditions file. You will need to set MHD_PIC_SubType parameter in initial conditions file or hard-code it into some startup routine. The numerical methods, tests, and first applications of this module are detailed in Ji, Squire, & Hopkins, arXiv:2112.00752 – any users using these physics should cite this paper.

Note that if this is active, by default, the code will assume there is one subtype/species of CRs, with a charge-to-mass ratio that can be specified in the parameterfile at runtime in the parameter PIC_Charge_to_Mass_Ratio (which defines this in dimensionless units of electron charge to proton mass, to a proton is unity). If you wish to specify this differently for different subtypes, its trivial to do so: just un-comment a line like the example in grain_physics.c, which reads: //if(grain_subtype==4) {charge_to_mass_ratio_dimensionless = -1836.15; /* electrons */, which gives an example for e.g. a simulation where MHD_PIC_SubType=3 corresponds to protons, and MHD_PIC_SubType=4 to electrons, to change the appropriate subtype and corresponding number assumed.

Also note that the default code implementations allow the PIC particles to be arbitrarily relativistic, but do assume that the background MHD fluid is non-relativistic.

PIC_SPEEDOFLIGHT_REDUCTION: Set this equal to a number to define a factor to reduce the speed-of-light for MHD-PIC simulations (relative to true value of c). If not set, the full value of c will be used. But in many applications, this can produce extremely small timesteps. As such a reduced-speed-of-light (RSOL) approximation is standard and allows for much longer timesteps (though the usual rule still applies, that the reduced speed of light should be faster than other signal speeds in the problem). Unlike many other implementations of an RSOL in PIC and MHD-PIC simulations, the formulation in GIMO, following Ji & Hopkins, arXiv:2111.14704 where it is rigorously derived, is mathematically guaranteed to produce the correct steady-state and equilibrium solutions for the distribution function of the evolved particles, and the correct values of all quantities such as energy, momentum, current, number, etc, of the PIC particles, as well as the correct forces on the gas, in any regime where the distribution function is close to steady state (not rapidly evolving). If this is used, cite Ji & Hopkins, arXiv:2111.14704

GRAIN_FLUID_AND_PIC_BOTH_DEFINED: This tells the code that both GRAIN_FLUID (integration of non-relativistic massive dust/particulate/aerosols particles) and PIC_MHD (for e.g. cosmic rays or other applications) are simultaneously active. cite Ji, Squire, & Hopkins, arXiv:2112.00752

Continuum Cosmic Ray-MHD Simulations

These flags enable and control simulations of a mixed fluid of ‘regular’ background fluid (which obeys whatever the desired equations-of-motion are, e.g. hydrodynamics, MHD, radiation-MHD, cooling/heating, elasto-dynamics) and populations of cosmic rays. The cosmic rays are assumed to be in the continuum limit – i.e. we evolve the distribution function of cosmic rays, as opposed to the trajectories of individual cosmic rays (for which you should use the MHD-PIC formulation). The formal derivation and justification of these methods is presented in Hopkins et al., arXiv:2103.10443. Essentially, the methods here evolve the exact equations for a gyrotropic population of cosmic rays, including all the leading-order terms in \(|{\bf u}|/c\), where \({\bf u}\) is the characteristic background/MHD fluid velocity (i.e. the CRs can be arbitrarily relativistic, but we assume the background fluid is non-relativistic) and \(r_{\rm gyro}/\ell_{\rm macro}\), the ratio of the gyro radius of the CRs to the macroscopic simulated scales. For e.g. a galaxy simulation with ~GeV CRs, both of these are excellent approximations (\({\bf u}\) is \(\sim 10^{-3}$c and $r_{\rm gyro}\) is \(\sim 0.1\) au). But of course there are some processes they cannot capture, like micro-scale plasma/kinetic processes and first-order CR acceleration. These are represented instead via some effective scattering rate and injection (in e.g. supernova remnants or shocks). Importantly, however, as shown in Hopkins et al., arXiv:2103.10443, the full two-moment formalism in GIZMO does not require or assume that the CR scattering mean-free path is short compared to other scales in the system (this is an important difference from some historical CR transport models). The scattering mean-free path can be arbitrarily long, i.e. the CRs can be free-streaming, with these modules turned on and treated appropriately – like with kinetic MHD, the key scale is their gyro-radii keeping them confined perpendicular to fields, not their mean-free-path.

The major methods papers for all of these are: Chan et al. 2019MNRAS.488.3716C, and Hopkins et al., 2020MNRAS.492.3465H, arXiv:2103.10443, and arXiv:2109.09762.

Some additional details of the modules are described in the notes_cosmicrays file (here), but a more formal and accurate description is in the methods papers cited above.

The specific flags include:

## ----------------------------------------------------------------------------------------------------
#COSMIC_RAY_FLUID               # top-level switch to evolve the distribution function (continuum limit) of a population of CRs. includes losses/gains, coupling to gas, streaming, diffusion. this uses the multi-moment formulation correct in arbitrarily anisotropic and free-streaming or fluid-like limits. all operators will be anisotropic as it should unless MHD is turned off. cite Hopkins et al. arXiv:2103.10443,2002.06211,2202.05283
#CRFLUID_DIFFUSION_MODEL=0      # determine how coefficients for CR transport scale. 0=spatial/temporal constant diffusivity (power law in rigidity), -1=no diffusion (but stream at vAlfven), values >=1 correspond to different literature scalings for the coefficients (see user guide). cite Hopkins et al. arXiv:2002.06211 and arXiv:2112.02153 for all derivations of the models here (and see refs therein for sources for some of the models re-derived and implemented here)
#CRFLUID_EVOLVE_SPECTRUM=2      # follow a spectrally-resolved CR population of (=1:e-/p, =2:e-,e+,p,anti-p,B,C/N/O,stable (7-9)Be, unstable 10Be) from ~MeV-TeV (and other species if extended network is enabled), including injection and adiabatic+hadronic/catastrophic/pionic/fragmentation+inverse compton+ionization+coulomb+bremstrahhlung+gyroresonant/streaming+synchrotron+annihilation+radioactive losses and (optionally) re-acceleration. cite Hopkins et al. arXiv:2109.09762 and arXiv:2112.02153 for methods and all numerical details of the implementation in any use (and Girichidis+ 2020MNRAS.491..993G for the fundamental spectral bin-to-bin method)
#CRFLUID_EVOLVE_SCATTERINGWAVES # follows Zweibel+13,17 and Thomas+Pfrommer 18 to explicitly evolve gyro-resonant wave packets which define the (gyro-averaged) CR scattering rates; requires MAGNETIC and COOLING for detailed MHD and ionization+thermal states. cite Hopkins et al. arXiv:2002.06211 for numerical implementation into various solvers here. 
#CRFLUID_ALT_RSOL_FORM          # enable the alternative reduced-speed-of-light formulation where 1/reduced_c appears in front of all D/Dt terms, as opposed to only in the flux equation. converges more slowly but accurately. both methods are derived and discussed in Hopkins et al, arXiv:2103.10443, which should be cited here
#CRFLUID_SPEEDOFLIGHT_REDUCTION=1 #  reduce the speed of light for CRs specifically (can be done separately from e.g. radiation), following the self-consistent formulation in Hopkins et al. arXiv:2103.10443, which should be cited for these methods
## ----------------------------------------------------------------------------------------------------

COSMIC_RAY_FLUID: This is a top-level switch needed for these modules: it must be enabled for the other modules here to work. If this is enabled but no other modules are, the CRs will be evolved in the simplest possible approximation: we will make the ‘single-bin’ approximation (so evolve only the CR total energy/pressure, aking to evolving a narrow bin of \(\sim 1-10\) GeV protons), assuming the CRs are ultra-relativistic (\(\gamma=4/3\)); and use the ‘single-moment’ or ‘zeroth-moment’ approximation, i.e. assume the CR flux equation is always in local steady-state with a nearly-isotropic distribution of pitch angles so that the CR energy density equation becomes a diffusion+streaming+advection equation; and will assume a constant-in-time-and-space scattering rate (hence effective diffusion coefficient). Assuming MAGNETIC is enabled, the transport even in this limit will be appropriately anisotropic along field lines, with the streaming speed assumed to be the Alfven speed (as appropriate for CRs in the self-confinement limit) with the effective diffusion coefficient (\(=v^{2}/3\,\bar{\nu}\) for a pitch-angle-averaged scattering rate \(\bar{\nu}\) and CR velocity \(v\)) set by hand in the parameterfile in code units with the variable CosmicRayDiffusionCoeff in that parameterfile. The correct anisotropic advection, streaming, and diffusion behaviors are automatically included, the cosmic rays appropriately act on the gas according to their parallel (scattering) and perpendicular (Lorentz) forces in the Riemann problem, the so-called ‘adiabatic’ heating/cooling terms are appropriately included modifying the CR energy, and assuming streaming is included, the ‘streaming-loss’ term is also calculated self-consistently (scaling roughly as \(v_{A}\,\nabla P_{\rm CR}\)). See e.g. Wentzel 1968, Skilling 1971, 1975, Holman 1979, as updated in Kulsrud 2005, Yan & Lazarian 2008, and Ensslin 2011. If COOLING is active, cosmic rays also self-consistently lose energy through Coulomb interactions, ionization, and catastrophic losses (in the single-bin approximation: if other approximations are used, more terms are included as appropriate), with the appropriate fraction of those terms going into heating the gas (see e.g. Volk et al. 1996, Ensslin et al. 1997, Mannheim & Schilickeiser 1994, and Guo & Oh 2008). It is straightforward to include CR injection in different sources – for example, in the FIRE models, CRs are injected into the gas via supernovae explosions (a fraction, typically \(\sim10\%\), of each SNe ejecta kinetic energy is assumed to go into the CR population). If MAGNETIC is off, i.e. there are no evolved magnetic fields in the code, then this module will still function, but will assume isotropic diffusion+streaming, with the Alfven speed replaced by the thermal sound speed (i.e. arbitrarily assuming plasma \(\beta=1\) everywhere): that is not really physical but may be useful for tests and experiments. The methods paper for the single-bin implementation here is Chan et al. 2019, MNRAS, 488, 3716 (arXiv:1812.10496), with first results, with updates to the method and additional tests presented in Hopkins et al. 2020MNRAS.492.3465H and arXiv:2103.10443.

CRFLUID_DIFFUSION_MODEL: This determines the model for the scattering rate coefficients used in cosmic-ray transport (which in turn determine the effective diffusivity and/or streaming speeds). It must be set to a valid integer, with each integer value corresponding to a different model. Default or =0 will set the diffusion coefficient to a constant in space and time, normalized to the value of CosmicRayDiffusionCoeff in code units (if you are evolving the entire CR spectrum, then this normalizes the constant power-law dependence of scattering rate on rigidity, \(\bar{\nu} = \bar{\nu}_{0}\,\beta\,(R/{\rm GV})^{-0.6}\), with \(\bar{\nu}_{0} = c^{2}/3\,\kappa_{0}\) where \(\kappa_{0}\) is the value set). If this variable is set to a value <0 (e.g. –1), then diffusion will be disabled and the CRs will be transported purely by streaming+advection. If set to a value >0, the code will attempt to use different literature models for how the diffusion and/or streaming coefficients should scale with local plasma properties. =1 gives the Jokipii 1966 classical extrinsic turbulence scaling for an isotropic, un-damped, extrinsic Kolmogorov cascade down to gyro-resonant wavelengths (note, as discussed in detail in arXiv:2112.02153, this is probably never valid, and if it is, it is only mathematically possible for this to be a valid model for scattering for CR energies much, much larger than TeV, and in many regimes larger than PeV). =2 is the Snodin et al. 2016 extrinsic random-field scaling (this also assumes an undamped isotropic cascade with similar power-spectra to the Kolmogorov model, and again could only possibly apply to extremely high-energy CRs). =3 uses the toy model Farber et al. 2018-like scaling, where the diffusivity is assumed to be fast in neutral gas, and slow in ionized gas – this is of course an ad-hoc model rather than one motivating a specific scaling rate, but can represent some potential physical effects discussed therein (the diffusivities are constant in space and time except for an ionization-fraction-dependent normalization, note). Model =4 is Wiener et al. 2017’s estimated equilibrium self-confinement motivated streaming speed for hot plasma (again as discussed in arXiv:2112.02153, this cannot capture, by definition, the correct energy dependence of CR scattering, but can be useful for numerical experiments in the single-bin approximation). =5 is the Chan et al. 2018 test of streaming at the fast magnetosonic speed (as stated therein this is not motivated by any actual physics, but just a numerical test). =6 will use the local steady-state scattering rates solving for the combination of extrinsic turbulence and gyro-resonant growth of modes in equilibrium with damping rates including turbulent, linear Landau, non-linear Landau damping, following Hopkins et al. arXiv:2112.02153. =7 combines the self-confinement scaling and extrinsic turbulence models for single-bin CR transport nominally favored in the parameter study of Hopkins et al 2021MNRAS.501.4184H (note it will not necessarily give physical answers for scattering coefficients for non-single-bin CR transport).

With this enabled and the flag set to =6 or =7, the behavior can be further customized with the special sub-flags CRFLUID_SET_SC_MODEL and CRFLUID_SET_ET_MODEL, which change the detailed assumptions in the model for the local steady-state scattering modes. Setting CRFLUID_SET_SC_MODEL to different values sets the mode parameter in the function diffusion_coefficient_self_confinement, where users can of course customize their behavior. Values =0, =1, =2, =3, =4 correspond to the ‘SC-default’, \(f_{\rm QLT}=100\), \(f_{\rm cas}=100\), \(f_{\rm cas-K41}\), \(f_{\rm cas-IK}\) models (respectively) for single-bin scattering explored in Hopkins et al. 2021MNRAS.501.4184H; setting to =5, =6, =7 correspond to the favored special modified damping rate \(\Gamma_{\rm damp,\,new}\), linear source term \(S_{\rm new,\,lin}\) and extrinsic source term \(S_{\rm new,\,ext}\) proposed and tested for full-spectrum CR scattering constraints in arXiv:2112.02153. Likewise setting CRFLUID_SET_ET_MODEL sets the mode variable in diffusion_coefficient_extrinsic_turbulence for extrinsic turbulence-dominated CR scattering: setting =-1 models a self-consistent Alfvenic cascade (any cascade which obeys a critical-balance type assumption or admits any mathematically-possible Alfvenic cascade between parallel/scattering modes; see the discussion in arXiv:2112.02153), as in Hopkins et al. 2021MNRAS.501.4184H for single-bin models. Setting =1 will use the Yan and Lazarian 2004 model for scattering by fast modes accounting for damping (again see discussion in both arXiv:2112.02153 and 2021MNRAS.501.4184H). Setting =0 will include both extrinsic turbulence terms. Other values are purely for numerical testing. Obviously the papers above should be cited if these are used, and more details can be found therein.

CRFLUID_EVOLVE_SPECTRUM: This will extend the CR equations to evolve an entire CR spectrum with multiple species, accounting self-consistently for their full spectral evolution, following Hopkins et al. arXiv:2109.09762 and arXiv:2112.02153. This is set to a value: if =1, a minimal set of species (just protons and electrons) are followed. If =2, an extended network including protons, electrons, anti-protons, positrons, B, C/N/O, stable (7 and 9)Be, unstable (10)Be, are all included. In both cases the spectrum span energies from MeV-TeV. In both cases, assuming MAGNETIC and COOLING variables are enabled, the network will include injection and adiabatic, hadronic/catastrophic/pionic/fragmentation, inverse compton, ionization, coulomb+bremstrahhlung, gyroresonant/streaming, synchrotron, annihilation, radioactive, diffusive re-acceleration, secondary (and tertiary and arbitrarily high-order) production terms – basically all known relevant CR loss/gain terms. The fundamental spectral bin-to-bin method follows directly from Girichidis et al., 2020MNRAS.491..993G, and the implementation and extensions to the various species and processes and coupling in GIZMO are exhaustively described and tested in Hopkins et al. arXiv:2109.09762 and arXiv:2202.05283, both of which should be cited if this is used. If this is on, note that the outputs will include not just the total CR energy in each spectral bin, but the local slope of the isotropic part of the CR distribution function, (optionally) the CR fluxes, and in the header, a large amount of information specifying the different species and information about the different spectral intervals (range of rigidities, species, many other details). Note that these will by default use the more accurate (as opposed to bin-centered) scheme for fluxes across the momentum range represented by each bin, as developed in Hopkins 2022 arXiv:2202.05283.

CRFLUID_EVOLVE_SCATTERINGWAVES: If this flag is active, the code will follow the methods proposed and outlined in Zweibel 2013 (2013PhPl…20e5501Z) and 2017 (2017PhPl…24e5402Z), and later Thomas and Pfrommer 2019 (2019MNRAS.485.2977T), to explicitly evolve the energy density in gyro-resonant ‘wave packets’ that define the (gyro and pitch-angle averaged) CR scattering rates. This is essentially a sub-grid model for the un-resolved small-scale scattering modes, that dynamically evolves the scattering rates with a time-dependence (as opposed to models with this off, which allow for complicated variability of these rates, but assume they are not changing if all other properties of the medium are held fixed to a single set of values). The specific equations solved, which extend the Zweibel and Thomas and Pfrommer methods to include various additional terms important in some regimes and a more general set of damping/driving mechanisms per the model set described above, are all given in Hopkins et al arXiv:2112.02153. Note this evolves both the forward and backward propagating mode wavepackets, so also allows for the interpolation between ET and SC limiting behavior for the Alfvenic streaming component. Note that this requires MAGNETIC and COOLING for proper thermo-chemistry and other quantities. This is still being tested and different assumptions experimented with for more extended applications to full-spectrum cosmic rays, the development version remains in flux as a result.

CRFLUID_ALT_RSOL_FORM: By default as noted above, the user can set a reduced speed of light (RSOL) to limit the maximum CR free-streaming speed and therefore allow larger timesteps. (Note, if the pure-diffusion approximation is used, the speed of light does not explicitly enter, so this is not used). Two different but in principle equally-valid formulations of the RSOL for the CR-MHD equations are derived and discussed in detail in Hopkins et al, arXiv:2103.10443, which should be cited here. Both methods are manifestly ensured to converge to the correct (true-speed-of-light) behavior when the CR distribution function is in local steady-state. But they have different behaviors when out-of-steady-state. By default (if this flag is not on), the code will use the formation of their Eq. 51, where the RSOL appears only in the time-derivative term for the CR flux equation (but not the CR energy equation). As discussed in detail therein, this has the advantage that it can converge to the correct steady-state solution more rapidly, especially in regimes where the diffusion/escape time for CRs is very large (e.g. the distant CGM from galaxies). However, this method does have some ambiguity when the CRs are dominated by certain loss terms and far away from steady-state (encapsulated by the \(\Psi\) function defined therein) that can produce some inaccuracies in detailed spectral models in special cases. Turning this flag on enables the alternative RSOL formulation, where the RSOL appears in just one power but in both the CR energy and flux equations. This completely removes the ambiguuities above, and so will converge more accurately in general. However, it will also converge more slowly in distant regimes: as a result, significantly higher values of the RSOL are recommended if this flag is active.

CRFLUID_SPEEDOFLIGHT_REDUCTION: This reduces the maximum free-transport speed of the CRs specifically in the lab/simulation frame - which would normally be the speed of light - by this factor (can be done separately from e.g. radiation), following the self-consistent formulation in Hopkins et al. arXiv:2103.10443, which should be cited for these methods.

This is developed by P. Hopkins, with major contributions from TK Chan for the CR-fluid and S Ji for the MHD-PIC implementations. The fluid modules solve the CR transport in the two moment limit (second-order expansion of the collisionless boltzmann equation) as in Hopkins et al. arXiv:2103.10443,2002.06211,2202.05283 (cite that paper for newest version, but also Chan et al. 2019MNRAS.488.3716C and arXiv:2002.06211 for initial code development of older formulations which paved the way for the implementation here). One can set the value here for the reduced speed of light (maximum free-streaming speed) in code units. All these modules require MAGNETIC for proper behavior for fully-anisotropic equations. Note this formulation with appropriate physical terms is valid in both the frequent-scattering and free-streaming limits, it does NOT require CRs be in the ‘fluid-like’ or frequent-scattering limit, only that resolved scales are large compared to CR gyro radii. For resolved scales smaller than CR gyro radii, use the PIC methods below.
More specifically, this will integrate the CR equations of motion using a two-moment expansion, i.e.\ solving for the energy and flux densities dynamically, loosely akin to the M1 approximation in radiation-hydrodynamics. The exact equations of motion are derived from the general Vlasov equation for an arbitrary gyrotropic distribution of CRs in Hopkins et al., arXiv:2103.10443, including all terms (as noted above) to leading order in \(|{\bf u}|/c\) and \(r_{\rm gyro}/\ell_{\rm macro}\), and valid in both the nearly-anisotropic and arbitrarily-anisotropic (free-streaming) limits of the CR distribution function. We stress that unlike the radiation M1 equations, as discussed in detail in arXiv:2103.10443, there is a rigorous basis for this expansion in terms of \(r_{\rm gyro}/\ell_{\rm macro}\) (i.e. gyrotropy is justified), as opposed to M1 RHD (where the closure is somewhat ad-hoc) – as such the range of regimes where this accurately captures the full behavior of CRs is extremely broad. If this is not turned on, then the single-moment diffusion+streaming equation will be used: that is only formally valid on scales large compared to the CR scattering mean free-path (which is relatively large, \(\sim 10\,{\rm pc}\,(R/{\rm GV})^{1/2}\)). But this will have no issue on scales either much shorter or much longer than that. This flag must be set to a value: the value given is the reduced speed of light (RSOL) for the CR free-streaming behavior specifically, in code units. The derivation of how the RSOL operates is again in Hopkins et al. arXiv:2103.10443: as shown therein, just like for RHD, this is implemented such that you are guaranteed to recover the exact solutions (with RSOL \(=c\)) for the DF so long as the system is in any local steady state or the CRs have traveled the same distance to some target, provided the RSOL is not slower than other characteristic speeds in the problem. Cite Hopkins et al. arXiv:2103.10443 (in addition to the other references above) if this module is used.

Note that various other special options, including disabling certain terms (cooling, losses, streaming), variable reduced speeds of light, different closure assumptions for comparison with older cosmic ray models, etc, are in the special/debugging options section of the code.

Sub-Grid Models for CR Effects on MHD Fluids

Here we include more simplified sub-grid or toy models which do not actually explicitly evolve CR populations at all, but simply represent them via some approximation in order to calculate their effects on the actually-explicitly-evolved fluid system.

## ----------------------------------------------------------------------------------------------------
#COSMIC_RAY_SUBGRID_LEBRON        #  uses a simplified sub-grid LEBRON-type model for CR transport for cheap approximation in galaxy simulations. Cite Hopkins et al. arXiv:2211.05811 for any use (see there for methods details)
## ----------------------------------------------------------------------------------------------------

COSMIC_RAY_SUBGRID_LEBRON: Turning this on will enable a simple sub-grid model for the CR population, which can then be used for CR pressure effects (adding the CR pressure to the Riemann problem, as in the tight coupling approximation) and simple heating terms from CR losses and streaming. The approximation is akin to the LEBRON RHD approximation in GIZMO, assuming some losses occur but only local to sources and then CRs propagate with a constant, spherically-symmetric independent steady-state solution for each source following a constant diffusion+effective streaming speed, as detailed and derived and tested in Hopkins et al. arXiv:2211.05811. That paper should be cited for any use of this code.

Turbulent ‘Stirring’ (Large Eddy Simulations)

These flags enable explicit turbulent ‘stirring’ of the simulation volume, as in ‘driven turbulence’ or ‘large eddy’ simulations. These are commonly used to study turbulence for its own sake, or set up initial or boundary conditions for many studies of the ISM and star formation.

####################################################################################################
# ------------------------------------- Driven turbulence (for turbulence tests, large-eddy sims)
# ------------------------------- users of these routines should cite Bauer & Springel 2012, MNRAS, 423, 3102, and thank A. Bauer for providing the core algorithms
####################################################################################################
#TURB_DRIVING                   # turns on turbulent driving/stirring. see begrun for parameters that must be set
#TURB_DRIVING_SPECTRUMGRID=128  # activates on-the-fly calculation of the turbulent velocity, vorticity, and smoothed-velocity power spectra, evaluated on a grid of linear-size TURB_DRIVING_SPECTRUMGRID elements. Requires BOX_PERIODIC
####################################################################################################

TURB_DRIVING: This activates a module, based closely on the original GADGET implementation shared by Andreas Bauer, to include a turbulent driving routine. It assumes a periodic box (BOX_PERIODIC on), which is then stirred as in Schmidt 2008, Federrath 2010, Price 2010: a small range of modes (which range set in the parameterfile) are driven in Fourier space as an Ornstein-Uhlenbeck process, with the compressive part of the acceleration optionally projected out via a Helmholtz decomposition in Fourier space so that the driving can range from purely compressive to purely incompressible/solenoidal (most appropriate for sub-sonic turbulence). The parameters of the driving are set in the parameterfile as described below. Users of this module should cite Bauer and Springel 2012, MNRAS, 423, 3102, as described above.

Note that in most driven turbulence experiments, it is common to ‘force’ the gas to lie along an exact adiabatic equation-of-state (a single adiabat or entropic function), so any energy of e.g. shocks and kinetic dissipation is immediately removed (if this or some cooling physics is not enabled, the turbulent driving will gradually ‘heat up’ the box, decreasing the mach number). This is accomplished as described above by turning on EOS_ENFORCE_ADIABAT in the Config file. For example, for a truly isothermal test, with isothermal sound speed equal to unity, you can simply set EOS_ENFORCE_ADIABAT=1 and EOS_GAMMA=1.0001 (this is slightly larger than unity, so the temperature is re-set nearly as if EOS_GAMMA=1 but various unphysical divergences that arise if GAMMA–1=0 will be avoided).

TURB_DRIVING_SPECTRUMGRID: This activates on-the-fly calculation of the turbulent velocity, vorticity, density, and smoothed-velocity power spectra; the power spectra are calculated over a range of modes and dumped to files titled powerspec_X_NNN.txt where NNN is the file number and X denotes the quantity the power spectrum is taken of (e.g. velocity, smoothed velocity, etc). The columns in these outputs are (1) k (Fourier mode number), (2) power per mode at k, (3) number of modes in the discrete interval in k, and (4) total discrete power over all modes at that k. To convert to a ‘normal’ power spectrum, and get e.g. the power per log-interval in k, take column (2) times the cube of column (1). The value to which you set TURB_DRIVING_SPECTRUMGRID determines the grid linear size (in each dimension) to which the quantities will be projected in taking the power spectrum. Users of this module should cite Bauer and Springel 2012, MNRAS, 423, 3102, as described above. Note that this only works if the box is periodic: BOX_PERIODIC must be enabled.

Gravity & Cosmological Integrations

####################################################################################################
## ------------------------ Gravity & Cosmological Integration Options ---------------------------------
####################################################################################################

These options all pertain to the gravity solver in the code. They determine how gravity is solved an allow for explicit modifications to the gravitional fields, periodicity of gravity, cosmological expansion history and integration, force softenings, and non-physical terms to prevent self-gravity in certain regimes (such as “artificial pressure” terms)

Numerical Solvers & Optimization

# --------------------------------------- TreePM Options (recommended for cosmological sims)
#PMGRID=512                     # adds Particle-Mesh grid for faster (but less accurate) long-range gravitational forces: value sets resolution (e.g. a PMGRID^3 grid will overlay the box, as the 'top level' grid)
#PM_PLACEHIGHRESREGION=1+2+16   # adds a second-level (nested) PM grid before the tree: value denotes particle types (via bit-mask) to place high-res PMGRID around. Requires PMGRID.
## -----------------------------------------------------------------------------------------------------
# --------------------------------------- Pure-Tree Options for Direct N-body of small-N groups (recommended for hard binaries, etc)
#GRAVITY_ACCURATE_FEWBODY_INTEGRATION # enables a suite: GRAVITY_HYBRID_OPENING_CRIT, TIDAL_TIMESTEP_CRITERION, to more accurately follow few-body point-like dynamics in the tree. currently compatible only with pure-tree gravity.
## -----------------------------------------------------------------------------------------------------
# --------------------------------------- Tree-Walk Precision (performance optimization)
#GIZMO_MIXED_PRECISION_GRAVITY  # use single-precision (float) for the gravity force-kernel accumulators and tree-node moment storage, while keeping particle positions in double precision. Cuts gravity-tree memory and bandwidth ~2x at sub-percent force accuracy cost for typical multi-physics simulations. Default OFF (all gravity quantities in double precision). Follows pkdgrav3 / Bonsai conventions.
##-----------------------------------------------------------------------------------------------------

PMGRID: This enables the TreePM method, i.e. the long-range force is computed with a PM-algorithm, and the short range force with the tree. The parameter has to be set to the 1-D size of the mesh that should be used, e.g. 64, 96, 128, etc. The mesh dimensions need not necessarily be a power of two, but the FFT is fastest for such a choice. Note: If the simulation is not in a periodic box, then a FFT method for vacuum boundaries is employed, using a mesh with dimension twice that specified by PMGRID.

PM_PLACEHIGHRESREGION: If this option is set (will only work together with PMGRID), then the long range force is computed in two stages: One Fourier-grid is used to cover the whole simulation volume, allowing the computation of the large-scale force. A second Fourier mesh is placed on the region occupied by ‘high-resolution’ particles, allowing the computation of an intermediate-scale force. Finally, the force on very small scales is computed by the tree. This procedure can be useful for ‘zoom-simulations’, where the majority of particles (the high-res particles) are occupying only a small fraction of the volume. To activate this option, the parameter needs to be set to an integer that encodes the particle types that make up the high-res particles in the form of a bit mask. For example, if types 0, 1, and 4 are the high-res particles, then the parameter should be set to PM_PLACEHIGHRESREGION = 19 or PM_PLACEHIGHRESREGION=1+2+16, i.e. to the sum \(2^{0} + 2^{1} + 2^{4}\) (sum of \(2^{n}\) over all particle types \(n\) desired). The spatial region covered by the high-res grid is determined automatically from the initial conditions. Note: If a periodic box is used, the high-res zone is not allowed to intersect the box boundaries.

GRAVITY_ACCURATE_FEWBODY_INTEGRATION: Normally, the default error tolerances in GIZMO’s tree-gravity solver will allow relatively large degradation of tight binary orbits if they form, because it is optimized for many-body dynamics with softened gravity (where individual resolution elements are not point-masses). If high accuracy in close few-body encounters is desired, a few flags will ensure this: several of these are rolled together into this convenience flag , which includes GRAVITY_HYBRID_OPENING_CRIT (uses both a Barnes-Hut and relative acceleration tree opening criterion, to be more conservative), LONG_INTEGER_TIME and STOP_WHEN_BELOW_MINTIMESTEP (to deal with deeper timestep hierarchies), and TIDAL_TIMESTEP_CRITERION to timestep carefully through close passages. However, this may drop the timesteps to very small values. To deal with this, the code modules in the SINGLE_STAR package allow for some major optimizations, developed by Mike Grudic. This module set is designed for simulations with ‘real’ point-particle or point-mass-like dynamics, as compared to smoothed gravity, so is only compatible with Tree-only gravity (as opposed to TreePM).

GIZMO_MIXED_PRECISION_GRAVITY: Enables a mixed-precision gravity tree walk that keeps particle positions in double precision but stores tree-node multipole moments (center of mass, total mass) and accumulates force-kernel sums in single precision (the typedef MyGravFloat, which is float when this flag is on and double otherwise, controls the relevant fields). This cuts the gravity-tree memory footprint and per-walk bandwidth by roughly \(\sim 2\times\) at a sub-percent typical force-accuracy cost for the regimes most multi-physics simulations actually probe (extended distributions with smoothed gravity), since the dominant error source in those regimes is the multipole-opening criterion rather than the floating-point precision of the accumulation. The narrowing cast happens at the moment-seed step from the underlying double-precision NODE structures, so the host tree build is unchanged. Follows the precision conventions of pkdgrav3 and Bonsai, where mixed precision is the default. Default OFF (all gravity quantities in double precision). This is primarily a performance/memory optimization for the GPU gravity tree walk; not recommended for very few-body close-binary integration (use GRAVITY_ACCURATE_FEWBODY_INTEGRATION instead, which keeps full double precision).

Adaptive Force Softenings

## -----------------------------------------------------------------------------------------------------
# ---------------------------------------- Adaptive Grav. Softening (including Lagrangian conservation terms!)
#ADAPTIVE_GRAVSOFT_FORGAS       # allows variable softening length (=KernelRadius) for gas particles
#ADAPTIVE_GRAVSOFT_FORALL=2     # enable adaptive gravitational softening lengths for designated particle types (ADAPTIVE_GRAVSOFT_FORGAS will be enabled). the softening is set to the distance
                                # enclosing a neighbor number set in the parameter file. flag value = bitflag like PM_PLACEHIGHRESREGION, which determines which non-gas particle types are adaptive (others use fixed softening). cite Hopkins et al., arXiv:1702.06148
#ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION=1 # enable adaptive gravitational softening lengths for designated particle types (value = bitflag like ADAPTIVE_GRAVSOFT_FORALL), but setting the softening to the tidal-tensor based value instead of nearest-neighbor values, following Hopkins et al. methods paper. this is cross-compatible with ADAPTIVE_GRAVSOFT_FORGAS, or even ADAPTIVE_GRAVSOFT_FORALL so long as the same particle type does not appear in both.
##-----------------------------------------------------------------------------------------------------

ADAPTIVE_GRAVSOFT_FORGAS: When this option is set, the gravitational softening lengths used for gas particles is tied to their kernel length/cell size. This is always recommended, since it means the gravitational equations (Poissons equation) is solved using the same mass distribution as the hydrodynamics. Otherwise, using different softenings for hydro and gravity, one obtains inconsistent solutions that many studies have shown will produce unphysical artifacts and may not converge. Unlike GADGET, the adaptive softening is done using the fully-conservative Lagrangian formulation similar to Price+Monaghan 2007 (modified for several important differences in e.g. how we calculate these lengths, and the fact that we use the larger softening in different-softening interacting pairs to symmetrize the forces which leads to much smaller integration errors for e.g. hard binaries and multiples), which prevents significant energy conservation errors.

ADAPTIVE_GRAVSOFT_FORALL: This uses adaptive kernel softenings for all desired particle types, including e.g. collisionless particles and not just gas. As with ADAPTIVE_GRAVSOFT_FORGAS, kernel softenings are set to some multiple of the distance to N nearest neighbors, again with appropriate conservation terms. This should be set to an integer bitflag value, like PM_PLACEHIGHRESREGION (see above): the integer encodes the particle types that make up the particles for which softenings are adaptive (other particle types will simply used the standard fixed-softenings that they would use without any ADAPTIVE_GRAVSOFT flag set). For example, if you want to allow types 0 (gas), 1, and 3 to have adaptive softenings (while types 2, 4, and 5 have fixed softenings), then the parameter should be set to ADAPTIVE_GRAVSOFT_FORALL = 2+8 or ADAPTIVE_GRAVSOFT_FORALL = 10, i.e. to the sum of \(2^{1}+2^{3}\) (the sum of \(2^{n}\), where each \(n\) is one desired particle type for adaptive softenings, noting that gas will automatically be activated with this flag via ADAPTIVE_GRAVSOFT_FORGAS). This flexibility in defining which species are adaptively-softened is handy for many applications where e.g. some particles really should not have adaptive softenings (e.g. sink particles, or stars, representing real point-like particles) while others (e.g. gas and dark matter) might represent volume-filling fluids. Enabling this adds the expense of a neighbor search for non-gas particles, and requires a Courant-like timestep condition for the collisionless particles (if they are adaptively softened), but allows greater accuracy in computing gravitational forces and much more flexible spatial resolution (since what is really fixed in an N-body code is the mass resolution). To avoid some redundancies and potential code conflicts/errors in the compilation, if this is enabled, ADAPTIVE_GRAVSOFT_FORGAS will be automatically enabled. The same kernel function is used for the adaptive force softening for other particle types as is used for gas (consistent with the hydrodynamic operators), be sure to set the desired neighbor number (in the parameterfile parameter AGS_DesNumNgb) appropriately (usually to the same number as used for the gas). In addition to the standard GIZMO methods paper, users of this module should cite Hopkins et al., arXiv:1702.06148, where the timestepping criteria and tests of this are developed.

By default, if a particle type is adaptively softened (part of this bitflag), then its softening is set by the inter-neighbor distance to particles of the same type (e.g. that fluid type is assumed to be volume-filling, so e.g. a lone gas particle would have a large softening, even if it has many dark matter particles with small softening around it). In the function ags_gravity_kernel_shared_BITFLAG you can, however, define any mutual interactions you like – for example, if you think particle types 2 and 3 should see one another when computing softenings, you can trivially specify this (see the notes therein). Some examples are pre-coded for specific flags: for example if you set ADAPTIVE_GRAVSOFT_FORALL to include stars in simulations with star formation, it will assume stellar softenings are set by the neighbor density of all baryons (gas and stars) rather than just other stars, to prevent newly-born stars from artificially jumping to large softening values (same with sink particles, if that physics is enabled). Another example: if DM_SIDM is set, particles of all valid SIDM types automatically see one another.

ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION: This flag acts somewhat like ADAPTIVE_GRAVSOFT_FORALL, in that it enables adaptive gravitational softening lengths for designated particle types, and you designate which particle types use this module by setting its value to a bitflag just like ADAPTIVE_GRAVSOFT_FORALL. The key difference is that while ADAPTIVE_GRAVSOFT_FORALL uses a scheme vaguely motivated by the gravitational softening which can be derived rigorously for the hydrodynamics (based on the density of neighboring cells, as if the particles were, like gas, a collisional, volume-filling, pressurized fluid) – even if the particles are collisionless so this assumption is not well motivated – the scheme ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION will instead employ to set the softenings is to adjust it based on the local value of the gravitational tidal tensor (or curvature tensor). The scheme follows Hopkins et al. (arXiv:2212.06851), and that paper should be cited for any use of it. As shown therein this has, for collisionless particles (e.g. stars, dark matter, dust) many advantages over the neighbor based scheme used by ADAPTIVE_GRAVSOFT_FORALL, especially when it comes to accurately and reliably evolving sub-structure. Note that this module is in fact cross-compatible with ADAPTIVE_GRAVSOFT_FORGAS (one can and often should keep ADAPTIVE_GRAVSOFT_FORGAS on for the gas, while enabling ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION for some subset of collisionless species). This will also use the rules derived in that paper for how to correctly and accurately handle interaction terms between different species that use different force softening rules. Any species no covered by the flags set will default to fixed in time-and-space softenings set in the run-time parameterfile.

When either ADAPTIVE_GRAVSOFT_FORGAS or ADAPTIVE_GRAVSOFT_FORALL or ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION is set, then the value of MaxKernelRadius in the parameterfile sets the maximum allowed value of the force softening (though this can be lower for other reasons, e.g. it cannot exceed the largest sizes of all gravity-tree-nodes), while the value of the force softening set for the adaptive particle types (e.g. controlled by Softening_Type0, Softening_Type1, or SofteningGas, SofteningHalo, etc) sets the minimum allowed value of the softening – these still need to be set but now they represent a minimum value (for adaptive particle types: recall, if the particle type is not one set to be adaptive by ADAPTIVE_GRAVSOFT_FORGAS or ADAPTIVE_GRAVSOFT_FORALL, then it will use a fixed softening, set to this value throughout the simulation, as usual). The minimum value is not simply zero, because there are many occasions where you want to limit it to prevent runaway self-gravitating collapse from e.g. producing un-resolved high densities or other un-desired behaviors. You can set it very small, though, in practice, for many applications: however try not to set it smaller than “floating point error” with respect to typical particle positions or interparticle separatations (e.g. if typical units of distance in code are 1, then setting SofteningHalo 1e-20 will lead to softenings that are much smaller than the floating point errors when doing things like calculating the separation (numerical difference of two position values) between two particles, which will (in turn) cause the code to think it is zero, which can lead to badness. But it is important that the minimum softening not be set to too large a value. If the value is too large, then many, many particles will appear under the same softening ‘kernel.’ This is fine with fixed (constant) softening, but not fine with adaptive softening, because the adaptive softening algorithm assumes the softening reflects the inter-neighbor separation between particles when calculating the correction terms from conservation. This assumption and the Lagrangian from which these terms are derived breaks down if you are forcing the softening to a much larger value (not to mention, you are paying all the costs of adaptive softening in terms of computation and timestep penalties, but not actually getting adaptive softening, since it is just ‘piling up’ at the minimum). That means you will artificially do bad things like lose energy upon integration, and can get unphysical effects. So set your softening limits carefully: a good rule-of-thumb for adaptively-softened particle types is to set this a factor of a few or ten smaller than the smallest inter-particle separations you want to allow in the simulation.

External Fields and/or Disabling Gravity

## -----------------------------------------------------------------------------------------------------
#SELFGRAVITY_OFF                # turn off self-gravity (compatible with GRAVITY_ANALYTIC)
#GRAVITY_NOT_PERIODIC           # self-gravity is not periodic, even though the rest of the box is periodic
## -----------------------------------------------------------------------------------------------------
#GRAVITY_ANALYTIC               # Specific analytic gravitational force to use instead of or with self-gravity. If set to a numerical value
                                #  > 0 (e.g. =1), then SINK_CALC_DISTANCES will be enabled, and it will use the nearest BH particle as the center for analytic gravity computations
                                #  (edit "gravity/analytic_gravity.h" to actually assign the analytic gravitational forces). 'ANALYTIC_GRAVITY' gives same functionality
##-----------------------------------------------------------------------------------------------------

SELFGRAVITY_OFF: This switches off gravity, for e.g. pure hydro simulations in non-expanding space.

GRAVITY_NOT_PERIODIC: Makes gravity non-periodic (when the overall box is periodic); useful for some special problems where you need periodic gas boundaries, but don’t want periodic gravity.

GRAVITY_ANALYTIC: Add a specific analytic gravitational force. The analytic function is specified in the file gravity/analytic_gravity.h, specifically in the routine add_analytic_gravitational_forces. Youll see a number of analytic functions already pre-defined there for you, but can always add more or change them. This can be used in addition to the standard self-gravity calculation (if e.g. you want to simulate just gas with its self-gravity in some fixed halo or other potential) or in addition to SELFGRAVITY_OFF if you wish to disable self-gravity but still impose some external potential (as in e.g. a Rayleigh-Taylor problem).

Non-Standard Dark Matter, Dark Energy, Gravity, or Expansion

##-----------------------------------------------------------------------------------------------------
#--------------------------------------- Self-Interacting DM (Rocha et al. 2012) and Scalar-field DM
#--------------------------------    use of these routines requires explicit pre-approval by developers J. Bullock or M. Boylan-Kolchin (acting for M. Rocha); approved users please cite Rocha et al., MNRAS 2013, 430, 81 and Robles et al, 2017 (arXiv:1706.07514).
#DM_SIDM=2                      # Self-interacting particle types (specify the particle types which are self-interacting DM with a bit mask, as for PM_PLACEHIGHRESREGION above (see description); previous "DMDISK_INTERACTIONS" is identical to setting DM_SIDM=2+4
#DM_SCALARFIELD_SCREENING       # Gravity is mediated by a long-range scalar field, with dynamical screening (primarily alternative DE models)
#DM_FUZZY=0                     # DM particles (Type=1) are described by Bose-Einstein Condensate: within gravity kernel (adaptive), solves quantum pressure tensor for non-linear terms arising from Schroedinger equation for a given particle mass. Still some testing recommended, while this is public we encourage discussion with PFH over different use cases and applications of the method. The value here is: 0=fully-conservative Madelung method. 1=mass-conserving direct SPE integration. 2=direct SPE (non mass-conserving). Cite Hopkins et al., 2019MNRAS.489.2367H if used.
#DM_HEATING                     # Continuous gas heating from DM annihilation and/or decay (rho_DM-driven source term added to DtInternalEnergy). Run-time params control the two channels independently. Cite Hopkins et al., 2026 (internal).
##-----------------------------------------------------------------------------------------------------
# -------------------------------------- arbitrary time-dependent dark energy equations-of-state, expansion histories, or gravitational constants
#GR_TABULATED_COSMOLOGY         # enable reading tabulated cosmological/gravitational parameters (top-level switch)
#GR_TABULATED_COSMOLOGY_W       # read pre-tabulated dark energy equation-of-state w(z)
#GR_TABULATED_COSMOLOGY_H       # read pre-tabulated hubble function (expansion history) H(z)
#GR_TABULATED_COSMOLOGY_G       # read pre-tabulated gravitational constant G(z) [also rescales H(z) appropriately]
##-----------------------------------------------------------------------------------------------------

​
DM_SIDM: Enables self-interacting dark matter, following M. Rocha, James Bullock, and Mike Boylan-Kolchin. The per-pair scattering probability is the kernel-weighted form given under “Scattering rate used for DM_SIDM and GRAIN_COLLISIONS” below; it replaces the tabulated overlap integral of the original implementation, and gives the same ensemble-averaged rate with a somewhat different pointwise normalization. Like PM_PLACEHIGHRESREGION, the parameter needs to be set to an integer that encodes the particle types that make up the SIDM particles in the form of a bit mask. For example, if types 1 and 2 are the SIDM particles, then the parameter should be set to DM_SIDM = 6 = 2 + 4 (i.e. \(2^{1} + 2^{2}\)). This allows for arbitrary combinations of SIDM particle types. Note that the legacy flag DMDISK_INTERACTIONS is now implicit in this (its identical to setting DM_SIDM=2+4). Please contact its developers (JB and MBK) before using the full module as written in the development branches in scientific work. But a slightly simpler version is part of the public code; moreover if users wish to use the implemented architecture and replace the actual self-interaction kernel with their own, they are free to do so provided they include the appropriate citations. If this is active, the user must set several parameterfile options that specify the dark matter self-interaction cross-section, its dependence on relative velocity, and the degree of exo-or-endo-thermality (i.e. dissipation or gain of kinetic energy per interaction). See the parameterfile description below for description of these. Users should cite Rocha et al., MNRAS 2013, 430, 81 and Robles et al, 2017 (arXiv:1706.07514).

Scattering rate used for DM_SIDM and GRAIN_COLLISIONS: both modules decide, for each neighbor pair, whether the two elements scatter in this timestep. The probability is built from the adaptive-softening kernel each element already carries, rather than from an overlap integral of two extended “clouds”. For self-interacting dark matter, with macro-particle masses M_i, M_j, cross-section per unit mass sigma/m (DM_InteractionCrossSection), relative velocity |dv|, timestep dt, and the usual kernel W(r,h):

p_ij = |dv| * dt * a^-3 * (sigma/m) * [ M_i*M_j / (M_i+M_j) ] * [ W(r,h_i) + W(r,h_j) ]

For grain-grain collisions the elements represent numbers of grains rather than masses: a particle of mass M and grain radius R stands for N = M/mu grains of mass mu = (4*pi/3) * rho_grain * R^3 (Grain_Internal_Density). Two particles of equal mass therefore hold very different numbers of grains if their grain sizes differ, and the collision rate is set by the side holding more. With hard spheres meeting across pi*(R_i+R_j)^2:

p_ij = |dv| * dt * a^-3 * [ N_i*N_j / (N_i+N_j) ] * pi*(R_i+R_j)^2 * [ W(r,h_i) + W(r,h_j) ]

Both expressions are symmetric, so the two elements of a pair compute the same probability and agree on whether they scatter, and an accepted event gives them equal and opposite momentum kicks. Summed over a particle’s neighbors, each one-sided term returns the density that particle moves through, so the rate is normalized by construction. Because W(r,h) vanishes for r >= h, the probability is non-zero exactly when r < max(h_i,h_j): the neighbor criterion is the ordinary symmetric one, with no widened search radius and no separate overlap test. The velocity-dependent suppression (DM_InteractionVelocityScale, Grain_InteractionVelocityScale) multiplies the expressions above unchanged.

DM_SCALARFIELD_SCREENING: Replaces the normal newtonian gravity by a screened scalar-field force. Look at the code if you want to get a better handle on or modify the scalar field parameters (these can trivially be made time-dependent, for example). Used for studying alternative gravity and dark matter theories – primarily alternatives to dark energy involving dynamical screening. Users should cite the GIZMO source code for implementation details (http://adsabs.harvard.edu/abs/2014ascl.soft10003H).

DM_FUZZY: DM particles (specifically those of Type=1) are treated as an ultra-light boson/scalar-field, i.e. Bose-Einstein Condensate (BEC) or scalar-field or ultra-light or fuzzy dark matter (FDM) models. This in fact can be used for evolving any Bose-Einstein condensate, if desired. Within the gravity kernel (gravity is automatically assumed to be adaptive, necessary for the volume domains over which the equations are integrated to be well-defined), this solves the non-linear Shrodinger-Poisson equation in the Madelung formulation (where the density represents the squared amplitude of the field, and the velocity is the local group velocity of the wavefunction). This looks like the collisionless Euler equations with the addition of a quantum pressure tensor. Details of the implementation are in Hopkins 2018 (arXiv:1811.05583). The module is public, it is currently in final testing stages, so users are encouraged to reach out to PFH to discess the different numerical sub-modules available here as detailed in the paper, and which might be the most useful for their particular problem (and where this method may or may not work well for different problems of interest).

DM_HEATING: Enables a continuous gas-heating source term driven by the local dark-matter density, modeling energy deposition from DM annihilation and/or DM decay into local baryons. The local DM mass density \(\rho_{\rm DM}\) at each gas cell is computed via the kernel sum already maintained by the dm_dispersion_loop (enabling DM_HEATING automatically activates the dispersion loop and its associated support fields CellP[].KernelRadiusDM, NumNgbDM, DM_Vx/y/z, DM_VelDisp, DM_Rho, regardless of whether GALSF_SUBGRID_WINDS is also on). The heating rate added to gas-cell specific internal energy is \(\dot{u}_{\rm DM} = f_h^{\rm ann}\,(\langle\sigma v\rangle/m_\chi)\,\rho_{\rm DM}^2\,c^2/\rho_{\rm gas} + f_h^{\rm dec}\,\Gamma\,\rho_{\rm DM}\,c^2/\rho_{\rm gas}\). The annihilation term uses the self-conjugate-DM convention with no extra factor of \(1/2\): the volumetric annihilation rate \(\tfrac{1}{2}n_\chi^2\langle\sigma v\rangle\) combined with rest-mass energy \(2 m_\chi c^2\) per event gives \(\rho_{\rm DM}^2\,(\langle\sigma v\rangle/m_\chi)\,c^2\) exactly. Users with Dirac/non-self-conjugate DM (separate \(\chi\), \(\bar\chi\) populations) should fold the appropriate factor of two into the supplied DM_AnnihilationSigmaV_over_mChi scalar. The heating is added to CellP[].DtInternalEnergy after the hydro step has zeroed and populated that field, so the cooling routine sees the combined hydro + DM-heating budget. Implemented as a continuous, deterministic source — no Monte-Carlo events, no daughter velocity kicks (those would be a future variant), no DM mass-loss bookkeeping (\(\Gamma t \ll 1\) assumed for any realistic decaying-DM run). Sterile-neutrino X-ray-line decay is a special case of the decay channel: set \(\Gamma\) from \((m_s, \sin^2 2\theta)\) and \(f_h^{\rm dec}\) from the photon-energy fraction. The four run-time parameters are described below. Setting either pair (annihilation \(\langle\sigma v\rangle/m_\chi\) + heating fraction; or decay \(\Gamma\) + heating fraction) to zero disables that channel; both pairs zero leaves the module compiled but inert (with a notice at startup). Cite Hopkins et al., 2026 (internal).

GR_TABULATED_COSMOLOGY: Top-level switch to enable cosmological integrations or regular simulations with non-standard, time-dependent dark energy equations-of-state, expansion histories, or gravitational constants. In this mode, the local gravitational forces will still be Newtonian, but the cosmological history will be different (and the gravitational constant G can, in principle, vary with time). This then allows the sub-switches GR_TABULATED_COSMOLOGY_W which will read the dark energy equation of state w(z) from a pre-tabulated file (specified in the parameterfile by “TabulatedCosmologyFile”). For a constant \(w(z)=w\), this just amounts to replacing the constant \(\Lambda \rightarrow \Lambda_{0}\,a^{-3\,(1+w)}\) in the Hubble function. For a time-varying \(w\), the expression used is the integral expression \(\Lambda \rightarrow \Lambda_{0}\,\exp{\{ \int_{1}^{a}\,\frac{3\,(1+w[a^{\prime}])}{a^{\prime}}\,d a^{\prime} \}}\). The parameter GR_TABULATED_COSMOLOGY_H will read the Hubble function H(z) itself from this file. GR_TABULATED_COSMOLOGY_G will also read the value of G(z), the gravitational constant (i.e. allows for time-varying gravitational constant, albeit with an “instantaneously” Newton force). Note that these options will not self-consistently alter the power spectrum of initial conditions. To use the same cosmological initial conditions (assuming these are chosen early enough so that linear growth theory applies) intending to get roughly the same \(z=0\) power spectrum, for example, one should change the starting scale factor (or redshift) in the parameterfile so that the linear growth factors match appropriately for the intended “initial time” – i.e. so \(g^{0}(z=0)/g^{0}(z_{\rm ICs}^{\rm original}) = g^{\prime}(z=0)/g^{\prime}(z_{\rm ICs}^{\rm new})\), where \(g^{0}\) is the growth factor in the “original” cosmology for which the ICs were built to be used starting at redshift \(z_{\rm ICs}^{\rm original}\) and \(g^{\prime}\) is the growth factor in the new, modified cosmology and \(z_{\rm ICs}^{\rm new}\) is the new starting redshift. Similarly, the initial velocity field should be re-scaled by the factor \([H^{\prime}(z_{\rm ICs}^{\rm new})\,\Omega^{\prime}(z_{\rm ICs}^{\rm new})^{0.6}]/[H^{0}(z_{\rm ICs}^{\rm original})\,\Omega^{0}(z_{\rm ICs}^{\rm original})^{0.6}]\). See the parameters description below for more details.

Continuum Vlasov (6D Phase-Space Integration) Methods

##-----------------------------------------------------------------------------------------------------
# -------------------------------------- Continuum Vlasov (6D Phase-Space Integration) Methods -- directly integrate the collisionless/weakly-collisional Boltzmann (Vlasov) equation in place of N-body sampling; paper in preparation, Hopkins 2026. Still under active development/testing; discuss with PFH before production use.
#CBE_INTEGRATOR=8               # top-level switch: represent the local velocity distribution function (DF) in each cell/particle as a Lagrangian mixture of this many basis functions (N_basis), evolved via Godunov-type fluxes between cells and bases instead of Monte-Carlo/N-body sampling. Designated particle types (CBE_INTEGRATOR_PARTICLETYPES) carry the representation; those types are automatically given adaptive gravitational softening (as ADAPTIVE_GRAVSOFT_FORALL). Cite Hopkins 2026 (in prep) if used.
#CBE_INTEGRATOR_PARTICLETYPES=2 # bitmask (as for PM_PLACEHIGHRESREGION above) of which particle types carry the CBE moment representation; default = type 1 (standard collisionless/DM type)
#CBE_INTEGRATOR_SECONDMOMENT    # evolve the second (stress) moment of each basis, i.e. the 10-moment scheme [mass, momentum, symmetric D(D+1)/2 stress tensor], giving each basis its own independent, anisotropic velocity dispersion rather than a single fixed effective softening/sound-speed. Recommended default whenever CBE_INTEGRATOR is used.
#CBE_INTEGRATOR_HEATFLUX        # extend to the 13-moment closure by additionally evolving a contracted third-moment (heat-flux) vector per basis. Implies CBE_INTEGRATOR_SECONDMOMENT. More accurate near-Maxwellian, but requires more careful realizability handling far from equilibrium; still experimental.
#CBE_INTEGRATOR_WITHGRADIENTS   # enable second-order, slope-limited (piecewise-linear) spatial reconstruction of each basis' primitive variables to the cell faces, instead of first-order (piecewise-constant) reconstruction. Strongly recommended for production accuracy.
#CBE_INTEGRATOR_RP_GAUSSIAN     # use the exact one-sided Gaussian-basis Riemann flux solution for inter-cell/inter-basis fluxes, instead of the default compact-support ("top-hat") approximate solver. Method-comparison option (not the production default).
#CBE_INTEGRATOR_COLLISIONS      # enable an intra-cell collision operator acting locally on the basis mixture within each cell (e.g. isotropic hard-sphere elastic scattering), allowing the method to interpolate smoothly between the collisionless and strongly-collisional (fluid) limits. Defaults to isotropic BGK-type relaxation (CBE_COLLISION_MODEL=CBE_COLLISION_MODEL_BGK_ISOTROPIC) unless CBE_COLLISION_MODEL is set explicitly. Requires setting the run-time parameter CBECollisionCrossSection>0 (default 0, i.e. collisions OFF/no-op even with this flag compiled in).
#CBE_INTEGRATOR_STRICT_FACE_SPD_GUARD  # additional (more expensive) guard strictly enforcing positive-definiteness of the reconstructed face-state stress tensor at every face, beyond the default realizability-repair operator. Safety/diagnostic option.
#CBE_INTEGRATOR_OUTPUT_MOREINFO # output additional per-basis diagnostic information in snapshots (enables OUTPUT_ADDITIONAL_RUNINFO)
#CBE_INTEGRATOR_OUTBUDGET_VERBOSE # per-basis outflow-budget ledger debug printf path (diagnostic only; verbose, not for production runs)
##-----------------------------------------------------------------------------------------------------

These flags implement a Lagrangian, finite-volume method for directly integrating the continuum collisionless (or weakly-collisional) Boltzmann/Vlasov equation in \(6+1\)-dimensional phase space, in place of the traditional Monte-Carlo/N-body sampling of point-mass “particles.” Each cell (mesh-generating point) carries a set of CBE_INTEGRATOR velocity-space basis functions – Gaussian-like components with independently evolved mass, mean velocity, and (optionally) anisotropic velocity dispersion – which are advanced in time via a Godunov-type scheme: at each face between neighboring cells, a basis on one side is paired with the most similar basis on the other side (by a Wasserstein-like velocity-space cost, with a “free-slot” fallback that routes mass into the least-populated basis when no good match exists), a one-sided Riemann-type flux is computed between the paired bases, and the mass/momentum/stress moments of each basis are updated conservatively. The method is designed to couple transparently to the existing \(N\)-body/tree gravity, adaptive softening, and domain-decomposition machinery in GIZMO – so a CBE_INTEGRATOR-active particle type still has a well-defined position and mass for gravity purposes, but its velocity-space content is a continuum DF rather than a \(\delta\)-function. This can represent multiple crossing streams, phase-mixed/thermalized distributions, and everything in between, with a modest, bounded cost multiplier relative to standard \(N\)-body methods (rather than the exponential blow-up of grid-based Vlasov solvers). A methods paper describing the algorithm, its validation on standard collisionless test problems, and example applications to dark matter, stellar dynamics, and weakly-collisional gas/dust is in preparation (Hopkins 2026); cite this if the module is used.

CBE_INTEGRATOR: Top-level switch turning on the CBE integrator for the particle types specified by CBE_INTEGRATOR_PARTICLETYPES. The value sets the number of basis functions \(N_{\rm basis}\) carried per particle/cell (e.g. =8); more bases can represent more complex, multi-modal, or strongly anisotropic DFs at increased memory and compute cost. Enabling this automatically implies adaptive gravitational softening for the same particle types (as ADAPTIVE_GRAVSOFT_FORALL), since the volume-partition on which the CBE fluxes are defined requires a well-defined, adaptive smoothing/interaction length. This module is still under active development and validation; discuss with PFH about the current status and appropriate use cases before using it in a production/publication run.

CBE_INTEGRATOR_PARTICLETYPES: Bitmask (identical convention to PM_PLACEHIGHRESREGION/DM_SIDM above) specifying which particle types carry the CBE basis-function representation. Defaults to type 1 (the standard collisionless/dark-matter type) if not set.

CBE_INTEGRATOR_SECONDMOMENT: Evolves the second moment (symmetric stress tensor) of each basis in addition to its mass and mean velocity – the “10-moment” scheme (mass + \(D\)-component momentum + \(D(D+1)/2\)-component symmetric stress, in \(D\) spatial dimensions). This lets each basis carry its own anisotropic velocity dispersion, rather than assuming a single width/softening for the whole DF, and is the recommended default for any use of CBE_INTEGRATOR.

CBE_INTEGRATOR_HEATFLUX: Extends the moment hierarchy to 13 moments by additionally evolving a contracted third-moment (heat-flux) vector for each basis (implies CBE_INTEGRATOR_SECONDMOMENT). This can improve convergence and accuracy for DFs close to a local Maxwellian, at the cost of requiring more careful realizability/positivity handling as the DF departs further from equilibrium (see the discussion of 10- vs. 13-moment trade-offs in the method paper). Still experimental.

CBE_INTEGRATOR_WITHGRADIENTS: Enables second-order spatial reconstruction: each basis’ primitive variables (density, mean velocity, and – if CBE_INTEGRATOR_SECONDMOMENT is on – stress) are extrapolated to the cell faces via slope-limited gradients (a Barth-\&-Jesperson-style limiter, component-separated between density, velocity, and stress to avoid one noisy component spuriously damping the others), rather than using the first-order, piecewise-constant cell-centered values directly. Strongly recommended for production runs; first-order (this flag off) is useful mainly for basic testing/debugging.

CBE_INTEGRATOR_RP_GAUSSIAN: Selects the exact one-sided Gaussian-basis Riemann flux solution for the inter-basis fluxes at each face, in place of the default compact-support (“top-hat”) approximate solver. The top-hat solver truncates the flux exactly at the signal speed and is the current production default (better-behaved for e.g. long-term dynamical/orbital tests); the Gaussian solver is provided primarily for direct method comparison.

CBE_INTEGRATOR_COLLISIONS: Adds an intra-cell (purely local, no inter-particle communication) collision operator that relaxes the basis mixture within each cell toward a more isotropic/thermalized state, allowing the same method to smoothly span the collisionless-to-strongly-collisional continuum (e.g. reproducing ordinary hydrodynamics in the strongly-collisional limit). Defaults to isotropic, elastic hard-sphere (BGK-type) relaxation if CBE_COLLISION_MODEL is not set explicitly to a different model. This flag only enables the operator’s machinery – it is a no-op (zero cross-section) until the run-time parameter CBECollisionCrossSection is set >0 (see below).

CBE_INTEGRATOR_STRICT_FACE_SPD_GUARD: Optional, more expensive guard that strictly enforces symmetric positive-definiteness of the reconstructed face-state stress tensor at every face (beyond the realizability-repair operator that is always applied after each conservative update). Primarily a safety/diagnostic option for stress-testing new configurations.

CBE_INTEGRATOR_OUTPUT_MOREINFO: Outputs additional per-basis diagnostic quantities to snapshots (automatically enables OUTPUT_ADDITIONAL_RUNINFO). Useful for debugging/validating the basis-function representation; not needed for normal production runs.

CBE_INTEGRATOR_OUTBUDGET_VERBOSE: Enables a verbose debug printf path for the per-basis outflow-budget ledger. Diagnostic only; produces substantial output and is not intended for production runs.

Artificial Pressure Floors

##-----------------------------------------------------------------------------------------------------
#EOS_TRUELOVE_PRESSURE          # adds artificial pressure floor force Jeans length above resolution scale (means you can get the wrong answer, but things will look smooth).  cite Robertson & Kravtsov 2008, ApJ, 680, 1083
##-----------------------------------------------------------------------------------------------------

EOS_TRUELOVE_PRESSURE: Adds a pressure term to the equations of motion following Truelove 1997, to prevent any fragmentation via self-gravity for which the “bottom scale” (the Jeans length) is smaller than the resolution limit. Note that this is commonly said to prevent “numerical fragmentation” but that is not really correct - the fragmentation it prevents is, in fact, physical (and should happen if we are solving the correct equations). Rather, this term artificially suppresses this fragmentation according to a desired threshold, so that only well-resolved structures appear in the simulation (even if those structures are physically incorrect). Therefore, the results using this criterion are inherently resolution-dependent, and it should be used with caution (it is not a ‘cure’ but a numerical side-step, and users should be careful). It can also tend to corrupt the temperature evolution of gas which is strongly dominated by the artificial pressure term. In any case, this is done in a manner that couples only to pressure, not to the internal energy or temperature of the gas (so the temperatures still becomes “correctly” cold, but artificial collapse below the resolution limit is suppressed). Users of this module should cite Robertson & Kravtsov 2008, ApJ, 680, 1083.

On-the-Fly Group (Halo, Galaxy, Cluster, etc) Finding

####################################################################################################
# --------------------------------------- On the fly FOF groupfinder
# ----------------- This is originally developed as part of GADGET-3 by V. Springel
# ----------------- Users of any of these modules should cite Springel et al., MNRAS, 2001, 328, 726 for the numerical methods.
####################################################################################################
## ----------------------------------------------------------------------------------------------------
# ------------------------------------- Friends-of-friends on-the-fly finder options (source in fof.c)
# -----------------------------------------------------------------------------------------------------
#FOF                                # top-level switch: enable FoF searching on-the-fly and outputs (set parameter LINKLENGTH=x to control LinkingLength; default=0.2)
#FOF_PRIMARY_LINK_TYPES=2           # bitflag: sum of 2^type for the primary type used to define initial FOF groups (use a common type to ensure 'start' in reasonable locations)
#FOF_SECONDARY_LINK_TYPES=1+16+32   # bitflag: sum of 2^type for the seconary types which can be linked to nearest primaries (will be 'seen' when calculating group properties)
#FOF_DENSITY_SPLIT_TYPES=1+2+16+32  # bitflag: sum of 2^type for which the densities should be calculated seperately (i.e. if 1+2+16+32, fof densities are separately calculated for types 0,1,4,5, and shared for types 2,3)
#FOF_GROUP_MIN_SIZE=32              # minimum number of identified members required to qualify as a 'group': default is 32
## ----------------------------------------------------------------------------------------------------
# -------------------------------------  Subhalo on-the-fly finder options (uses "subfind" source code).
## ----------------------------------------------------------------------------------------------------
#SUBFIND                            # top-level switch to enable substructure-finding with the SubFind algorithm
#SUBFIND_ADDIO_NUMOVERDEN=1         # for M200,R200-type properties, compute values within in this number of different overdensities (default=1=)
#SUBFIND_ADDIO_VELDISP              # add the mass-weighted 1D velocity dispersions to properties computed in parent group[s], within the chosen overdensities
#SUBFIND_ADDIO_BARYONS              # add gas mass, mass-weighted temperature, and x-ray luminosity (assuming ionized primoridal gas), and stellar masses, to properties computed in parent group[s], within the chosen overdensities
## ----------------------------------------------------------------------------------------------------
#SUBFIND_REMOVE_GAS_STRUCTURES      # delete (do not save) any structures which are entirely gas (or have fewer than target number of elements which are non-gas, with the rest in gas)
#SUBFIND_SAVE_PARTICLEDATA          # save all particle positions,velocity,type,mass in subhalo file (in addition to IDs: this is highly redundant with snapshots, so makes subhalo info more like a snapshot)
####################################################################################################

These flags enable group finding via an on-the-fly FOF finder which is run at intervals set by the run-time parameters. This is most commonly used for halo finding in cosmological simulations, to build on-the-fly halo catalogues, or to simply track halos for purposes of seeding sink particles and other on-the-fly physics. However it has also been modified in previous applications to track things like star clusters, planet-forming clumps, or really any type of self-gravitating objects on-the-fly. The current defaults assume halo-finding, however, by defining structures as above a traditional virial overdensity. Users of any of these modules should cite Springel et al., MNRAS, 2001, 328, 726 for the numerical methods.

FOF: Top-level switch required for all modules in this block (as well as any SUBFIND implementation). The default FOF linking length is 0.2 times the mean inter-particle separation of the desired type. This can be adjusted by adding the compiler parameter LINKLENGTH set to some desired value.

FOF_PRIMARY_LINK_TYPES: This sets the primary particle type for the linking/group-finding. For dark matter, this is typically particle-type 1, which means setting the parameter to 2. The parameter is set as a bit-wise value (see notes in the gravity section)

FOF_SECONDARY_LINK_TYPES: This is the list of particle types (set bit-wise again) which should be linked to the primaries, i.e. other particle types that should be included in the FOF group-building and associated with different halos/groups.

FOF_DENSITY_SPLIT_TYPES: The list of particle types (set bit-wise again) which should have separately-calculated densities for linking purposes. If e.g. you are halo-finding in a cosmological simulation with star formation, this may need to separate dark matter and stars that have wildly different density distributions, or it will give spurious results.

FOF_GROUP_MIN_SIZE: Minimum size for a saved FOF group (in terms of total particle number). Set as desired.

Sub-Structure (Subhalo, Satellite, etc) Finding

SUBFIND: Top-level switch to enable substructure-finding with the SUBFIND algorithm from Springel et al., MNRAS, 2001, 328, 726 (cite this if using these algorithms). This identifies local density maxima within an FOF group, then moves outwards to associate material with that maxima while it is part of a ‘peak’ (with criteria to decide whether and how to ‘merge’ subgroups at saddle points), followed by an iterative un-binding criterion so that only bound sub-structures are saved. This is originally designed for detecting sub-halos (and associated satellite galaxies) in cosmological simulations, and saving their properties on-the-fly (that can be done in post-processing, of course, but this was intended to allow much more frequent monitoring and saving of these properties). But variants of this algorithm have been used for a variety of structure-finding tasks, including molecular cloud, star cluster, and planetesimal detection. The search is executed alongside the FOF search, whenever snapshots are written (allowing you to specify arbitrary times for writeouts: of course, full snapshots can be compressed or deleted after these writeouts, if they are set very frequent for storage reasons), and by default the outputs are saved in snapshot-like HDF5 files which are numbered alongside the snapshots and contain the group lists, with some gross properties of the group, their membership in the FOF groups, and the complete list of all particle IDs which are members of each group/substructure. These can be used with the snapshots to fully reconstruct the substructures. Additional output options are below (these are off by default because they can always be reconstructed in post-processing).

In the parameterfile, you must set the parameter DesLinkNgb, which is the neighbor number used for density estimation and creation of linked-lists to determine the substructure membership, saddle-points, etc. Usually this is set to the same value as DesNumNgb or a a couple times larger, if additional smoothing (to eliminate spurious resolution-scale structures) is desired, but in special cases it can be very different. Note that run-time tests and de-bugging may still be needed for some SUBFIND modules (users: please contribute notes or pushes if you are using these), although they have all been tested to compile properly with all combinations of other modules in GIZMO and basic functionality should work well. Note that the SUBFIND modules remain fundamentally coded in the old GADGET–3 style, meaning they are not multi-threaded, and do not take advantage of the modular parallelism structures, etc (they have not been re-written in the more modern style mainly because the groups using them have not found they are a major expense). Also note that by modern standards, SUBFIND is a relatively ‘simple’ algorithm for sub-structure finding: its primary utility is when users want something which can run on-the-fly in the simulations. There are a large number of extremely sophisticated sub-structure finders (which involve a large number of additional features and options) which can be run on GIZMO snapshots in post-processing: if you are interested, see the section of this guide on Halo/Group-Finding and Structure Identification.

A very similar version of this algorithm (also adapted from the original GADGET3 implementation of SUBFIND) has recently been made public in the public release of AREPO (Weinberger, Springel, and Pakmor, arXiv:1909.04667): users are encouraged to cite that paper for more technical details, as well as the Springel et al., MNRAS, 2001, 328, 726 algorithm paper, if they are using this module. All credit goes to Volker Springel for these algorithms.

SUBFIND_ADDIO_NUMOVERDEN: Set this to a number to compute FOF group quantities (masses, radii, and all properties set with the SUBFIND_ADDIO_... flags below) within that number of different ‘thresholds’ in terms of over-density. Default is =1, i.e. just one density, taken to be (1) \(200\bar{\rho}\) (200 times the mean background matter density). Higher values (=2, =3, etc., up to =10) will add (2) the spherical top-hat overdensity for collapse (accounting for full cosmological parameters), (3) 200 times the critical density, (4) 500x critical, (5) 1000x critical, (6) 2500x critical, (7) 500x mean, (8) 1000x mean, (9) 2500x mean, (10) 5000x mean. The output values will be vectors of length SUBFIND_ADDIO_NUMOVERDEN, corresponding to the values for each threshold in the order above. You can trivially change this list by editing Delta_ToEvalList in subfind_so.c.

SUBFIND_ADDIO_VELDISP, SUBFIND_ADDIO_BARYONS: Enable these flags to add additional quantities to the saved sub-structure properties. Each will be evaluated inside of every radius defined by the overdensity list of SUBFIND_ADDIO_NUMOVERDEN, i.e. like mass and radius above, these will be saved as vector outputs with each value corresponding to the value inside the relevant overdensity criterion. SUBFIND_ADDIO_VELDISP saves the mass-weighted mean (sightline-averaged) 1D velocity dispersion (of all elements) inside the sub-structure. SUBFIND_ADDIO_BARYONS adds some baryonic properties including gas mass, stellar mass (assuming stars have Type=4), mass-weighted gas temperature and X-ray luminosity (calculated assuming a fully-ionized primordial-composition gas, with X-rays from bolometric thermal Bremsstrahlung). These modules have been more updated, compared to the rest of the SUBFIND code, to be templated so that users can easily add their own additional quantities to calculate and output in SUBFIND catalogues. Simply search for one of the relevant flags (e.g. SUBFIND_ADDIO_BARYONS) to see the 5 or so places in the code where it appears, where the new physics desired can be modularly added (basically the user needs to define the variables to be computed/passed, and define the actual operation on resolution elements which will compute them, then decide which, if any, to write to file. for the writing, see the results of that search, but be sure to follow all the places the relevant output flag appears: for an example see e.g. SIO_DELTA_MGASSUB).

SUBFIND_REMOVE_GAS_STRUCTURES: Delete (do not save) any structures which are entirely gas (or have fewer than target number of elements which are non-gas, with the rest in gas). Useful for some applications where there may be a lot of dense gas sub-clumps which you do not want to store (if you are primarily interested in e.g. dark matter halos). This is primarily for storage efficiency.

SUBFIND_SAVE_PARTICLEDATA: Save all particle positions, velocities, types, and masses in the SUBFIND output file, for each member associated with a given substructure. Normally, only the ID-list of particles/cells associated with a given substructure are saved to allow you to combine with the snapshot files to re-construct any other quantities in post-processing. This makes the SUBFIND outputs essentially a snapshot onto themselves (and users could easily add additional fields saved), but obviously requires much larger storage.

Galaxy & Galactic Star Formation Options

This is a large set of flags which control the physics of galaxy-scale star formation. Here star formation refers to the formation of stellar populations, not the explicit resolution of individual stars and explicit resolution of their masses. That is handled by a different set of flags below. Several of the following flags are in active development, so please be careful, and contact their developers before using them.

#################################################################################################### 
#-------------------------------------- Galaxy formation and galactic star formation
##-----------------------------------------------------------------------------------------------------
#GALSF                           # top-level switch for galactic star formation model: enables SF, stellar ages, generations, etc. [cite Springel+Hernquist 2003, MNRAS, 339, 289]
####################################################################################################

First, there is the ‘top-level switch’ for galaxy formation and star formation.

GALSF: Enables star formation, tracking of ‘star particles’ and their associated quantities (ages, etc). Required for everything else in this block. Requires COOLING on. These follow the algorithmic star formation model wherein gas elements are individually converted into star particles (on a probabilistic basis, from a smooth continuum star formation rate) as originally described in Springel & Hernquist, 2003, MNRAS, 339, 289 (and that paper should be cited for their use). The different specific star formation and (especially) stellar feedback sub-routines below have their own citations and developers, described with each.

Star Formation Criteria/Particle Spawning

#################################################################################################### 
## ----------------------------------------------------------------------------------------------------
# --- star formation law/particle spawning (additional options: otherwise all star particles will reflect IMF-averaged populations and form strictly based on a density criterion) ---- #
## ----------------------------------------------------------------------------------------------------
#GALSF_SFR_CRITERION=(0+1+2)     # mix-and-match SF criteria with a bitflag: 0=density threshold, 1=virial criterion, 2=convergent flow, 4=local extremum, 8=no sink in kernel, 16=not falling into sink, 32=hill (tidal) criterion, 64=Jeans criterion, 128=converging flow along all principle axes, 256=self-shielding/molecular, 512=multi-free-fall (smooth dependence on virial), 1024=adds a 'catch' which weakens some kinematic criteria when forces become strongly non-Newtonian (when approach minimum force-softening), 2048=time-averaged virial
#GALSF_SFR_VIRIAL_SCALING=2   # instead of a threshold, implements a semi-continuous SF efficiency as a function of alpha_vir. set 0=step function between 1 and 0.01; 1=Padoan 2012 prescription; 2=multi-free-fall model, as in e.g. Federrath+Klessen 2012/2013 ApJ 761,156; 763,51 (similar to that implemented in e.g. Kretschmer+Teyssier 2020), based on the analytic models in Hopkins MNRAS 2013, 430 1653, with correct virial parameter
#GALSF_SFR_IMF_VARIATION         # determines the stellar IMF for each particle from the Guszejnov/Hopkins/Hennebelle/Chabrier/Padoan theory. Cite Guszejnov, Hopkins, & Ma 2017, MNRAS, 472, 2107
#GALSF_SFR_IMF_SAMPLING          # discretely sample the IMF: simplified model with quantized number of massive stars. Cite Kung-Yi Su, Hopkins, et al., Hayward, et al., 2017, "Discrete Effects in Stellar Feedback: Individual Supernovae, Hypernovae, and IMF Sampling in Dwarf Galaxies". 
#GALSF_GENERATIONS=1             # the number of star particles a gas particle may spawn (defaults to 1, set otherwise if desired)
## ----------------------------------------------------------------------------------------------------------------------------
####################################################################################################

These parameters control the star formation criteria (used to determine when and how star particles are spawned from gas) and algorithmic aspects of the star particle ‘spawning’. Use of these specific modules is permitted, as described below, as part of the use of the development code, and users are of course encouraged to modify things like the detailed criteria for spawning star/sink particles based on their specific applications; however proper citations should be included in any published work, to the papers which developed these physics and numerical models.

GALSF_SFR_CRITERION: This is the preferred flag to set the criteria for star-particle formation (rather than setting individual flags in older versions of GIZMO). You set the value to a bitflag which allows users to specify any arbitrary combination of criteria for star formation (provided they are physically compatible): the bitflag is set to the sum of the values corresponding to each criterion you want to enable, including: (0) A density threshold (set by the CritPhysDensity parameter in the params file). Note because the value is 0 here, this is effectively always on, but you can essentially disable it by simply setting CritPhysDensity very low or zero. (1) A local virial criterion (self-gravitating at the resolution scale including kinetic, thermal, and magnetic energy (see the description below for details of what this entails and for the proper references if you use it). (2) Requires the gas be in a convergent flow (\(\nabla\cdot{\bf v}<0\)). (4) Require the gas cell sit at a local density and potential extremum (evaluated within kernel). (8) Require that there are no other sinks inside three times the gas kernel/neighbor radius, and the gas is not a valid ‘interacting neighbor’ of any other sink particle – note this is usually only useful in individual-star-formation (sink-particle-type) simulations, but can be useful in galaxy scale simulations (where sinks represent supermassive black holes) in some circumstances. (16) Require the timescale for the gas element to free-fall onto (using a 2-body orbit) or intercept (\(\sim 1/(\Delta{\bf v}\cdot \Delta {\bf r})\)) the nearest sink is longer than the local dynamical collapse time – again this is primarily for single-star sink-particle simulations. Cite Grudic et al. arXiv:2010.11254 for the algorithm. (32) Enforce a Hill-type criterion that the local tidal tensor is negative definite (forces are converging along all three principle axes) and that the gas cell lives inside its own Hill sphere. This is a much stronger and more rigorous (Galilean-invariant) version of a local maximum-type criterion. Cite Grudic et al. arXiv:2010.11254 for the algorithm. (64) Require that the thermal+magnetic Jeans mass be less than the minimum of 1000 solar masses or the gas cell mass (but no smaller than 100 solar masses). This effectively requires that the gas can fragment down to plausible star-forming scales. Cite Grudic et al. arXiv:2010.11254. (128) Require that the flow be converging along all three principle axes by diagonalizing the velocity shear tensor. This is a much stricter version of the converging-flow criterion, though not necessarily always more physical. Cite Grudic et al. arXiv:2010.11254. (256) Require that the gas be self-shielding (molecular) and scale the star formation rate to the molecular fraction in the gas cell. (512) Scale the star formation efficiency per free-fall time not to a constant (the default behavior), but to a multi-free-fall model based on the simulations in Federrath+Klessen 2012/2013 ApJ 761,156; 763,51 (similar to that implemented in e.g. Kretschmer+Teyssier 2020), based on the analytic models in Hopkins MNRAS 2013, 430 1653, with correct virial parameter dependence. (1024) Adds a purely-numerical ‘catch’ where the more complicated kinematic criteria are ignored if the gas kernel size approaches the minimum force softening, where the forces become non-Keplerian and so some of the criteria above (like the virial criteria) are fundamentally ill-defined. (2048) Uses a time-averaged version of the virial criterion (see below).

For example, adding the bitflag ‘256’ to this specifically enables molecular-gas dependent star formation. This estimates the molecular fraction \(f_{H2}\) in a gas particle, and corrects the calculated star formation rate by this factor (only allowing star formation from the molecules). It uses whatever function the cooling routines are also using to estimate the molecular fractions (see details on this below in the various COOL_MOLECFRAC... options), as a function of e.g. local column density, metallicity, radiation field, etc. In evolved galaxies, for a high density threshold for SF (e.g. \(n> 100\,{\rm cm^{-3}}\)), the molecular fractions are very near unity so this has no effect, but it costs almost nothing to add this criterion to simulations, and it can be useful in lower-resolution simulations to approximate regions where gas can self-shield and cool. Any publications using this module should cite both the relevant reference for where the molecular fraction estimator is taken, and Hopkins et al., 2017a, arXiv:1702.06148 (Appendix C describes the algorithm in detail). Because of the dependence on cooling physics and metallicity, this requires METALS and COOLING be enabled.

For any of the virial criterion-based models, self-gravity is calculated via the local virial parameter following Hopkins, Narayanan, & Murray 2013 (arXiv:1303.0285); see that paper for details. This is useful for high-dynamic range simulations because it is adaptive - so even if the mean galaxy density is much larger than your SF threshold, this will only allow SF in the bound sub-regions. However, it can be expensive, since it forces you to resolve those (often very high-density) regions. Physically, this corresponds to the results of idealized turbulent cloud simulations and observations of actual star forming regions (see e.g. Padoan et al.), which show bound clouds have SF efficiencies of ~100% per free-fall time, while unbound clouds have negligible SF. Any publications using this module (or SF criteria above equal to =1, or =512, or =2048, etc.) should cite both Hopkins, Narayanan, & Murray 2013 (arXiv:1303.0285) and Hopkins et al., 2017a, arXiv:1702.06148 (Appendix C describes the algorithm in detail).

Calculating the virial parameter according to a running time-smoothed average (exponential window smoothed over the free-fall time) will reduce noise (particularly useful for star cluster or small-scale sink-particle simulations). The option to require a convergent flow along all three principle axes diagonalizing the shear tensor, can be made even stricter with the Hill-type criterion where we calculate tidal tensor and diagonalize it, requiring a convergent acceleration along all three principle axes with the resolution element size within its own Hill sphere. For any of these options, cite Grudic et al. arXiv:1708.09065.

GALSF_SFR_VIRIAL_SCALING: This flag allows you to modify the behavior of the virial star formation threshold, implementing a semi-continuous star formation efficiency per free-fall time as a function of the virial parameter, instead of a simple threshold/pure step-function value, in star formation. It requires you have appropriate values of GALSF_SFR_CRITERION set so that a virial criterion is being calculated/used in the first place. If this is set to different values, different models will be used. If GALSF_SFR_VIRIAL_SCALING=0, then the code will use a step function at \(\alpha_{\rm vir}=1\) between a star formation efficiency of unity (normalized to whatever is set in the parameter file, so this can be arbitrarily large or small) for \(\alpha_{\rm vir}<1\) to zero for \(\alpha_{\rm vir}>1\). If GALSF_SFR_VIRIAL_SCALING=1, then this will use the model from Padoan 2012, with SFE \(=\exp{(-1.4 \alpha_{\rm vir}^{1/2})}\). If GALSF_SFR_VIRIAL_SCALING=2, then this will use the multi-free-fall model from Federrath and Klessen 2012 ApJ 761,156; and 2013 ApJ 763,51 (the model implemented in e.g. Kretschmer and Teyssier 2020), which scales based on the analytic models in Hopkins MNRAS 2013, 430 1653, with the correct virial parameter. If GALSF_SFR_VIRIAL_SCALING=3, will adopt a step function from unity to one percent instead. Cite these papers when the appropriate model is used. Caution is needed here: the Padoan and Hopkins/Federrath+Klessen models have some regimes where they were not intended to apply and can give unphysical results in these regimes: for example predicting normal star formation in laminar, hot gas with a Toomre \(Q\gg 1\) if the disk phase structure is not extremely well-resolved, or predicting very high efficiencies in ultrasonic outflows around supernova remnants.

GALSF_SFR_IMF_VARIATION: Self consistently computes the IMF for each star particle from its parent gas properties at the time of formation. This uses the Hopkins/Guszejnov/Hennebelle/Chabrier model, if desired, but the default is a universal IMF. The IMF is then used self-consistently for feedback calculations throughout the simulation. This will also save and dump in snapshots a large number of properties of the star-forming clouds from which every star particle formed, for use in IMF models. This is currently in development by David Guszejnov and PFH, please contact us if you wish to use the module, as it is still in early testing phases and, while modular, requires the IMF scalings desired be input in a specific form. Users should cite Guszejnov, Hopkins, & Ma 2017, MNRAS, 472, 2107.

GALSF_SFR_IMF_SAMPLING: Enables stochastic sampling from the IMF. When a star particle forms, a discrete integer number of massive stars is chosen based on its mass and IMF. These appropriately scale the feedback properties and are lost in SNe. This is potentially important when the particle masses are very small, so that one would otherwise be associating a small fraction of an O-star to each star particle. Note that this module, in isolation, won’t do anything. Your stellar evolution/feedback modules must be appropriately coded so that they will use this flag to correctly parse the individual sampling. Users should cite Kung-Yi Su, Hopkins, et al., Hayward, et al., 2017, “Discrete Effects in Stellar Feedback: Individual Supernovae, Hypernovae, and IMF Sampling in Dwarf Galaxies”.

GALSF_GENERATIONS: This sets the number of ‘generations’ or independent star particles which can be spawned from a single gas particle. If not set, it defaults to 1. If set to an integer larger than 1, each gas particle can spawn multiple stars. Useful in some rare cosmological circumstances, but using values >1 tends to make simulations much more expensive and more poorly load-balanced, so use with caution.

Sub-Grid Feedback Models

#################################################################################################### 
## ----------------------------------------------------------------------------------------------------------------------------
# ---- sub-grid models (for large-volume simulations or modest/low resolution galaxy simulations) -----------------------------
# -------- the SUBGRID_WINDS models are variations of the Springel & Hernquist 2005 sub-grid models for the ISM, star formation, and winds.
# -------- Volker has granted permissions for their use, provided users properly cite the sources for the relevant models and scalings (described below)
#GALSF_EFFECTIVE_EQS            # Springel-Hernquist 'effective equation of state' model for the ISM and star formation [cite Springel & Hernquist, MNRAS, 2003, 339, 289]
#GALSF_SUBGRID_WINDS            # sub-grid winds ('kicks' as in Oppenheimer+Dave,Springel+Hernquist,Boothe+Schaye,etc): enable this top-level switch for basic functionality [cite Springel & Hernquist, MNRAS, 2003, 339, 289]
#GALSF_SUBGRID_WIND_SCALING=0   # set wind velocity scaling: 0 (default)=constant v [and mass-loading]; 1=velocity scales with halo mass (cite Oppenheimer & Dave, 2006, MNRAS, 373, 1265), requires FOF modules; 2=scale with local DM dispersion as Vogelsberger 13 (cite Zhu & Li, ApJ, 2016, 831, 52)
#GALSF_WINDS_ORIENTATION=0      # directs wind orientation [0=isotropic/random, 1=polar, 2=along density gradient]
#GALSF_FB_TURNOFF_COOLING       # turn off cooling for SNe-heated particles (as Stinson+ 2006 GASOLINE model, cite it); requires GALSF_FB_THERMAL
## ----------------------------------------------------------------------------------------------------
####################################################################################################

These flags relate to popular ‘sub-grid’ models for star formation and feedback. These models replace explicit resolution of the ISM with simple analytic model approximations, and “tuned” models for galactic winds designed to reproduce the stellar mass function. These are useful for comparison studies or for large-volume cosmological simulations where the resolution is ~100pc - kpc, but more explicit ISM physics, wind, and star formation models are also available in the code for small-scale simulations.

GALSF_EFFECTIVE_EQS: This enables the ‘effective equation of state’ (EOS) model originally developed by Springel & Hernquist, MNRAS, 2003, 339, 289. We point the interested reader to that paper for the detailed description (and that paper should be cited, if this is used). This replaces explicit resolution of a multi-phase ISM (as well as any cooling below \(\sim 10^{4}\) K) with a stiff EOS for gas above some modest density (a single-phase polytropic gas), motivated by a McKee & Ostriker-like analytic toy model for a two-phase ISM. The parameters describing this EOS are specified in the parameterfile. This is the model used in GADGET–2/3 and AREPO, and in many major simulation projects (“Illustris”, “OWLS”, “Eagle”, the Oppenheimer & Dave IGM studies, and more). Note that this is a module for the rather complex physical model for the effective EOS developed therein; if you simply want to insert a pure-polytropic EOS above some density, this can be accomplished by instead adding a single line in the get_pressure subroutine in “eos.c” (see what is done with the EOS_ENFORCE_ADIABAT or EOS_TRUELOVE_PRESSURE flags, for example). Volker Springel originally developed this module for GADGET–3, but he has allowed us to make it public in the current GIZMO form (with slightly modified implementation), on the understanding that users will properly credit the papers that actually developed these modules. Users should specifically cite Springel & Hernquist, MNRAS, 2003, 339, 289.

GALSF_SUBGRID_WINDS: This enables sub-grid “decoupled” winds: essentially, for every unit mass stars formed, some proportional mass is forced out of the galaxy at a desired velocity, with the “mass loading” set by hand in the parameterfile. As above, this is the ‘base’ numerical model used in GADGET–2/3 and AREPO, and in many major simulation projects (“OWLS”, “Eagle”, “Illustris”, etc). One can choose to allow the winds to be hydrodynamically de-coupled, or immediately recoupled, using parameterfile options. Users should cite Springel & Hernquist, MNRAS, 2003, 339, 289.

GALSF_SUBGRID_WIND_SCALING: The value of this parameter determines the scalings for the sub-grid wind model employed (requires GALSF_SUBGRID_WINDS). (0) Fixed (constant) wind velocity and mass-loading. Users should cite Springel & Hernquist, MNRAS, 2003, 339, 289. (1) Prescription developed by Oppenheimer & Dave, in which the halo mass (calculated in the on-the-fly friends-of-friends algorithm, which must be enabled), to launch the winds at the escape velocity with a mass-dependent mass-loading designed to reproduce the observed stellar mass function. Users should cite Oppenheimer & Dave, 2006, MNRAS, 373, 1265. (2) Uses the local DM velocity dispersion (kernel-computed around gas, no FOF required) to estimate escape velocity (otherwise acts like model “1”) and mass-loading. This is the method used in the Illustris simulations (Vogelsberger et al.), and implemented in GIZMO by Q. Zhu. Users should cite Zhu & Li, ApJ, 2016, 831, 52.

GALSF_WINDS_ORIENTATION: Enabling this forces the sub-grid winds to have a desired orientation. 0=isotropic (the particle “kicks” are uniformly randomly oriented), 1=polar (the particle kicks are directed along the direction given by the cross of the particle velocity and acceleration, which should be the polar axis in a disk), 2=along the local density gradient. If not set, and the wind model does not otherwise have a default preference built-in, the default will be isotropic. Users should cite the appropriate paper for the winds model they have adopted/implemented (see notes above).

GALSF_FB_TURNOFF_COOLING: This enables the ‘blastwave’ or ‘cooling turnoff’ model developed by Stinson et al. 2006 in GASOLINE. SNe occur (this requires GALSF_FB_THERMAL is on), and when they do, a fixed fraction of their energy is coupled thermally to the gas, which is then not allowed to cool for some long timescale (set analytically following Stinson et al. 2006), typically \(\sim 10^{7} - 10^{8}\) years (much longer than the physical cooling time of the gas). This is the model used a in wide range of GASOLINE projects (ERIS, MAGICC, etc). Users should cite Stinson et al. 2006 for the physical prescriptions, and consult PFH when using this module (the implementation in GIZMO has been tested under some conditions, but turning off cooling can produce strange interactions with cooling and heating routines under various circumstances so must be treated with caution).

Explicit (Resolved) Feedback Models

#################################################################################################### 
## ----------------------------------------------------------------------------------------------------------------------------
# ---- explicit thermal/kinetic stellar models: i.e. models which track individual 'events' (SNe, stellar mass loss, etc) and inject energy/mass/metals/momentum directly from star particles into neighboring gas
# -------- these modules explicitly evolve individual stars+stellar populations. Event rates (SNe rates, mass-loss rates) and associated yields, etc, are all specified in 'stellar_evolution.c'. the code will then handle the actual injection and events.
# -------- users are encouraged to explore their own stellar evolution models and include various types of feedback (e.g. SNe, stellar mass-loss, NS mergers, etc)
#GALSF_FB_MECHANICAL            # explicit algorithm including thermal+kinetic/momentum terms from Hopkins+ 2018 (MNRAS, 477, 1578): manifestly conservative+isotropic, and accounts properly for un-resolved PdV work+cooling during blastwave expansion. cite Hopkins et al. 2018, MNRAS, 477, 1578, and Hopkins+ 2014 (MNRAS 445, 581)
#GALSF_FB_THERMAL               # simple 'pure thermal energy dump' feedback: mass, metals, and thermal energy are injected locally in simple kernel-weighted fashion around young stars. tends to severely over-cool owing to lack of mechanical/kinetic treatment at finite resolution (better algorithm is mechanical)
##-----------------------------------------------------------------------------------------------------
####################################################################################################

These models attempt to explicitly model stellar feedback – e.g. track individual SNe events and directly model continuous stellar mass loss, associating the return into neighboring gas elements with the results of individual stellar evolution tracks or stellar population tracks. They can be directly enabled here, and also form ‘core’ underlying parts of the algorithm for more extensive specific feedback models below (e.g. the FIRE models or the SINGLESTAR models).

For both GALSF_FB_MECHANICAL and GALSF_FB_THERMAL, you (the user) should specify the SNe or other mechanical feedback properties, including event rates and the mass/energy/velocity/metal content of each ‘event’ in the routines mechanical_fb_calculate_eventrates and particle2in_addFB_fromstars which are both defined in stellar_evolution.c. If you do not specify anything, the code will default to a very simply model from the AGORA project (Kim et al., 2016 ApJ, 833, 202), assuming your star particles have an IMF-averaged number of core-collapse SNe which all occur at once (in a single timestep) when the particle reaches 5 Myr age, and each SNe injects the identical mass (14.8 solar) energy (1e51 erg) and metal mass (2.6 solar masses with solar abundance ratios). But this is very easy to customize – you can add stellar wind mass loss, neutron star mergers, Ia supernovae – you just have to specify the rates and ‘loading’ parameters (e.g. mass, momentum, energy, metal content) being released at a given timestep, given your star particle properties. The code modules will handle all the actual numerics of the injection.

The flags here are focused on mechanical-type feedback (e.g. SNe, stellar mass-loss, kilonovae, with associated mass/metal/energy/momentum), as opposed to radiative feedback specifically. However they can be trivially modified to represent local radiation injection or injection of other species such as cosmic rays. If you want to propagate radiation (or cosmic rays) long-distance, of course, you need to turn on the appropriate radiation (or cosmic ray) transport modules.

You must choose one of the two following modules, depending on whether you just want to inject a pure-scalar quantity, without regard for resolving the work done by expanding blastwaves or winds (in which case the simpler ‘thermal dump’ algorithm is sufficient) or whether you wish to handle vector quantities (e.g. momentum) and ensure isotropy and conservation and proper handling of un-resolved expansion phases (in which case the mechanical module is the correct choice).

GALSF_FB_MECHANICAL: This enables the explicit mechanical (e.g. SNe, stellar winds, etc) feedback coupling algorithm described in detail in Hopkins et al. 2018, MNRAS, 477, 1578. This is a sophisticated algorithm which ensures manifest energy and momentum conservation, and statistical isotropy, even with arbitrary particle arrangements around the ‘injection’ site. As shown in that paper, this is non-trivial: simply depositing momentum with a kernel-weighting around the source leads to large violations of linear momentum and energy conservation which can seriously corrupt predictions for galaxy or star cluster evolution; moreover in quasi-Lagrangian codes this almost always leads to all the ejecta being ‘dumped’ in the direction of the highest densities (e.g. into thin disks, as opposed to ‘venting’) which is physically the opposite of the correct solution. Equally or more important, this algorithm also explicitly accounts for whether or not the code resolves (or partially-resolves) the reverse shocks and Sedov-Taylor (energy-conserving) phase of the ejecta. It properly accounts for the conversion of energy into momentum (e.g. the PdV work) done by the ejecta reaching the resolved radii where it is coupled, and for un-resolved cooling. As many studies have shown, including Hopkins et al 2014 (MNRAS 445, 581, where this algorithm was first introduced), and subsequently Kimm and Cen 2014 (ApJ 788, 121), Martizzi et al 2015 (MNRAS 450, 504), Rosdahl et al 2017 (MNRAS 466, 11), and others, this produces the correct solutions at the actual resolved radii of the blastwaves (compared to high-resolution simulations of those blastwaves in uniform media with properties identical to the cell where they explode), and so has stable and orders-of-magnitude superior convergence properties compared to models in which the ejecta energy is simply coupled as ‘pure thermal’ or ‘pure kinetic/momentum’ (with some arbitrary specify-able efficiency). Any users of this module should cite Hopkins et al. 2018, MNRAS, 477, 1578 (and ideally the earlier studies above, as well).

GALSF_FB_THERMAL: This implements a simple “pure thermal energy dump” feedback: mass, metals, and thermal energy are injected locally in simple kernel-weighted fashion around young stars. The kernel follows a typical SPH-like density kernel, so the deposition is effectively mass-weighted (as opposed to e.g. a more accurate solid-angle weighting). This is an extremely simplified local model for stellar feedback, not a model for mechanical feedback (which, critically, must include the actual momentum and solve for wind/SNe shock properties at the resolved interface with the ISM – this module ignores all of that, which is done instead by completely different routines in mechanical_fb.c). As is well-known, owing to the neglect of these terms, such implementations will tend to over-cool (rather severely) at the resolution of typical galaxy simulations (for demonstrations, see Hopkins et al., arXiv:1707.07010). However, this implementation is potentially useful either (a) at extremely high mass resolution (particle masses \(\ll 100\,M_{\odot}\)), or (b) for historical comparison, as many models in the literature use this.

FIRE Project Explicit Feedback Models

#################################################################################################### 
## ----------------------------------------------------------------------------------------------------
# ----- FIRE simulation modules for mechanical+radiative FB with full evolution+yield tracks (Hopkins et al. 2014, Hopkins et al., 2017a, arXiv:1702.06148 and 2203.00040) ------ ##
# -------- Use of these modules as part of the public code is now allowed with appropriate citations to the specific methods papers above. These should be referred to as using the methods "from the FIRE public code (as in citations)", not as FIRE collaboration papers or FIRE simulations. FIRE simulations/papers follow FIRE collaboration guidelines, and you should reach out to members of the collaboration if you wish to write FIRE papers or use still in-development FIRE codes/outputs/etc.
#FIRE_PHYSICS_DEFAULTS=3        # enable standard set of FIRE physics packages (number=fire version). note use policy above. convenience flags for MHD (FIRE_MHD), BHs (FIRE_BHS), and CRs (FIRE_CRS=X, where X=-2 is toy-model sub-grid LEBRON-approximation CRs; -1=single-bin CRs (defaults to constant scattering rate); 0=full-spectrum p+e, constant (power-law) scattering rate; 1=full-spectrum p+e, variable scattering rate 2509.07104; 2=full-spectrum 10-species treatment)
#FIRE_SUPERLAGRANGIAN_JEANS_REFINEMENT #- super-lagrangian refinement based on jeans mass or other criteria. this is a generic flag to be used for high-resolution massive-galaxy simulations using hyper-refinement to achieve 'standard' high-resolution FIRE quality in galaxies, without trillion-particle loads
#GALSF_FB_FIRE_RT_HIIHEATING    # enable the generic stochastic HII-region model when HII regions are not extremely well-resolved, ionize and heat to local equilibrium temperature in immediate vicinity of O-stars probabilistically so they can be semi-resolved in a statistical sense. cite FIRE methods papers. do not use if enabling FIRE defaults flags OR any explicit RHD modules of your own (FIRE modules will work with any other RHD modules/bands where they are enabled)
#GALSF_FB_FIRE_RT_LONGRANGE     # enables LEBRON radiative transport with continuous acceleration from starlight (uses luminosity tree) to propagate approximate RT, also enables simple uv/euv/nuv/optical/nir/fir band set, for photo-ionization, photo-electric, and dust radiation heating. do not use if enabling FIRE defaults flags OR any explicit RHD modules of your own (FIRE modules will work with any other RHD modules/bands where they are enabled)
#GALSF_FB_FIRE_RT_LOCALRP       # enables local radiation pressure coupling to gas - account for local multiple-scattering and isotropic local absorption, assuming unresolved local-source approximation, designed to be used when those are not treated explicitly (i.e. if and only if an RHD module like LEBRON is used, as opposed to M1 or some other RHD module. do not use if enabling FIRE defaults flags OR any explicit RHD modules of your own (FIRE modules will work with any other RHD modules/bands where they are enabled)
#GALSF_FB_FIRE_AGE_TRACERS      # add explicit tracking of tracer species for arbitrary stellar yields as a function of age from different stellar populations. cite Wetzel et al. 2023 (arXiv:2203.00040)
##-----------------------------------------------------------------------------------------------------
####################################################################################################

These flags control the feedback models used in the series of FIRE papers, explicitly tracking of multiple forms of feedback physics (without free or tuned parameters). Papers presented as FIRE simulations or FIRE collaboration papers follow the FIRE collaboration’s own policies (projects pre-approved and posted to the collaboration, authorship offered, collaboration citation policies followed, etc.); for versions of these modules still in development, please contact the FIRE core development team (PFH, D. Keres, C.A. Faucher-Giguere, and E. Quataert) first. When using these modules, you should cite the appropriate FIRE methods papers, namely Hopkins et al. Hopkins et al. FIRE1/2/3 methods: arXiv:1311.2073, 1702.06148 and 2203.00040. Using those modules in your own work does not constitute a FIRE simulation or collaboration paper, you should refer to these numerical methods as following the implementation from FIRE in (cite appropriate FIRE version papers), as provided in the public GIZMO code, similar to citation policy for public FIRE simulation snapshots. If you wish to join the FIRE collaboration to participate in the FIRE project, write FIRE papers, and access non-public datasets, please reach out to members of the collaboration to discuss joining.
The numerical details and physics are described in great detail in Hopkins et al., 2017a, (arXiv:1702.06148) and Hopkins et al., 2017b (arXiv:1707.07010). Both of these papers should be cited in any publication using these modules (in addition to following the FIRE collaboration citation policies).

FIRE_PHYSICS_DEFAULTS: Enables the FIRE modules, default values for all below (see FIRE wiki for more details on recommended settings and compiler flags for FIRE simulations). FIRE simulations always should use exactly the same code-base and code options – make sure you consult the FIRE wiki to be sure you are using the correct frozen code version, flags, parameter-settings, etc, so that your run is exactly the same source code and options as all other FIRE runs (all FIRE runs requre this). If this is enabled no other FIRE flags below should be used.

Note there are many sub-flags not explicitly listed in the template Config file, because we prefer they are enabled as part of the default package physics. But we list them here for completeness, so it is possible to turn desired flags on and off in more detail.

GALSF_FB_FIRE_STELLAREVOLUTION: Enables explicit tracking of gas mass recycling and energy deposition from stellar winds (OB and AGB) and individual SNe Types Ia (with a fitting function for both prompt and delayed components from Manucci et al. 2006) and II (rates from Starburst99), with stellar evolution tracks, rates, yields, kinetic luminosities all tabulated from stellar evolution models. Rates are continuous and fully time and metallicity-dependent; for SNe at each timestep the number of SNe associated with a given particle is drawn from a Poisson distribution with the appropriate expectation value. For winds mass is moved continuously from star particles to nearby gas, together with the appropriate metal, momentum, and energy content, according to the time, metallicity, and mass-dependent yields from all wind sources in Starburst99. So at early times, for example, when O-star winds dominate, the returned material is very high energy, but later (when AGB winds dominate), it is much lower-energy. This requires GALSF_FB_MECHANICAL in order to work properly (set automatically if FIRE_PHYSICS_DEFAULTS is enabled).

GALSF_FB_FIRE_RPROCESS: This will make the code track an additional set of “dummy” elements (which do not have any dynamical effects). The motivation was to study a wide range of models for R-process element enrichment via neutron-star mergers, but this could easily be used to study any passive scalars in the ISM. The value GALSF_FB_FIRE_RPROCESS should be set to an integer giving the number of dummy species. The default GALSF_FB_FIRE_RPROCESS=4 will follow the default code for 4 different models of R-process enrichment (each of the 4 follows a different delay-time distribution). The additional metals will be appended to the usual metallicity array in the snapshot, following after the “usual” elements in order. For the default implementation here, the assumed enrichment for each species is (0) rate = 1e–5 event/Msun/Myr after tmin=3e7 yr, zero before, and elements shared to all neighbors, (1) tmin=3e6 yr, rate=1e–5, (2) tmin=3e7 yr, rate=1e–6, (3) tmin=3e6 yr, rate=1e–6. This flag requires GALSF_FB_MECHANICAL to work properly.

GALSF_FB_FIRE_RT_LONGRANGE: Enable long-range radiation pressure effects. Specifically, enabling GALSF_FB_FIRE_RT_LONGRANGE calculates the luminosity of each star particle (in standard fashion from Starburst99) and feeds it into the gravity tree as a repulsive \(1/r^{2}\) force, with the ‘strength’ at each point determined by the relevant absorption cross-section (integrated over three crude wavelength intervals in UV, optical, and IR - thats an approximation to a full spectrum, which compared to the other uncertainties here is actually quite accurate). The relevant code is in the “forcetree” code. Because its incorporated into the gravtree structure, it incurs very little cost and has no problems with the tree being spread across multiple processors, etc. The SED shape emerging from each local region around each star particle is calculated self-consistently, by calculating the ‘initial’ emission as a function of stellar age (the Starburst99 spectrum) and then attenuating it with the opacity estimated from the local density and density gradients. In this case, the parameters PhotonMomentum_fUV/PhotonMomentum_fOPT set a minimum escape fraction from each star particle in these wavelengths. When on, the escape fraction of far and near-UV photons from the local region around each star particle (with their UV spectra modeled from Starburst99) is calculated (integrating the local density and density gradients according to the Sobolev approximation), then the escaped photons are propagated through the gravity tree similar to the luminosity in GALSF_FB_FIRE_RT_LONGRANGE. For each gas particle they then intercept, the self-shielding around that particle and by intervening material along the line of sight is calculated with a second Sobolev approximation at the location of each particle encounter. The ‘surviving’ flux is then saved and used in the cooling function to calculate self-consistently the long-range contribution to photoionization heating from stars in the galaxy (as opposed to just the UV background), and to calculate photo-electric heating rates following Wolfire et al. 1995.

GALSF_FB_FIRE_RT_LOCALRP: This module controls the local momentum-deposition from radiation pressure within dense gas surrounding stars (including any possible multiple-scattering of IR photons). If GALSF_FB_FIRE_RT_LOCALRP is on, the code will search around all active star particles and follow a simplistic radiation transport calculation by assuming a photon momentum \(~(1-\exp(-\tau_{\rm opt,uv}))\,(1+\tau_{\rm IR})\,L/c\) within the kernel is potentially imparted, where \(\tau_{\rm IR}\) is the local IR optical depth explicitly calculated from the gas surface density and metallicity. Formally what the code uses is an integral over the flux equation with a differential acceleration per unit mass; so it is identical to an exact gray-opacity radiative transfer calculation if the gas is smooth on the kernel scale. By default, the sources are star particles, as described above.

GALSF_FB_FIRE_RT_HIIHEATING: Enables local photo-ionization heating by young star particles. This calculates the total number of ionizing photons per unit time from Starburst99 for each star, and then makes a stromgren approximation – moving outward from the star, it ionizes every un-ionized particle until it runs out of photons. those particles are flagged so that they cant cool below some minimum temperature \(\sim 10^{4}\) K (set in the parameterfile) for the rest of the timestep (and are heated to that temperature if already below it). This is not meant to mock up cosmological photo-ionization, and in fact will cut off before it gets to the relevant distances (for speed/stability reasons) – it doesnt include effects like scattering or redshifting out of resonance or finite light travel time that are necessary for those scales.

GALSF_FB_FIRE_AGE_TRACERS: Add explicit tracking of tracer species for arbitrary stellar yields as a function of age from different stellar populations, developed by Andrew Emerick (methods paper Wetzel et al. 2023, in prep, in the meantime cite arXiv:2203.00040). enabled by default with the FIRE defaults for newer FIRE versions.

GALSF_FB_FIRE_AGE_TRACERS: Enables explicit tracking of tracer species for arbitrary stellar yields as a function of age from different stellar populations. When enabled, additional passive scalar fields are appended to each particle’s metallicity array, each corresponding to a discrete stellar age bin. As star particles age, the feedback (mass, energy, metals) deposited by each age bin is tracked separately in nearby gas. By default, the age bins are logarithmically spaced between AgeTracerBinStart (default 1 Myr) and AgeTracerBinEnd (default 14000 Myr), with AgeTracerEventsPerTimeBin (default 10) controlling the deposition rate normalization. If GALSF_FB_FIRE_AGE_TRACERS_CUSTOM is also set, a custom list of bin edges can be specified via AgeTracerListFilename. This is enabled by default with FIRE_PHYSICS_DEFAULTS for newer FIRE versions. Users should cite Wetzel et al. 2023 (arXiv:2203.00040).

FIRE_SUPERLAGRANGIAN_JEANS_REFINEMENT: This enables a flexible set of super-resolution dynamic refinement models. This will be used for high-resolution simulations of massive halos, to ensure a target mass resolution in galaxies without refining everywhere, which would require an infeasible number of cells, despite most of those being IGM or hot halo gas which requires relatively coarse resolution for purposes of modeling feedback (there are other reasons of course to want high resolution in these regions, but we cannot practically refine everywhere)

GALSF_ISMDUSTCHEM_MODEL: Enables live ISM dust evolution, tracking the formation, destruction, and processing of multiple dust species self-consistently with stellar evolution, feedback, and gas-phase chemistry. The value is a sum of bitflags selecting which dust species are followed: 1=silicates and carbonaceous dust, 2=metallic iron dust, 4=oxygen-bearing dust species (an oxygen reservoir), 8=metallic iron nanoparticles carried as inclusions within silicates. Bit 1 is the base set and is always required – silicates and carbon are tracked in every variant, and omitting it is a compile-time error – and bit 8 requires bit 2, since the iron inclusions are drawn from the metallic iron. Formation channels include stellar sources (SNe Type Ia/II, AGB winds) with composition-dependent yields; destruction channels include sputtering in hot gas and destruction by SNe shocks; growth channels include gas-phase accretion in the ISM. A typical choice is GALSF_ISMDUSTCHEM_MODEL=(1+2). Users should cite Choban et al. 2022 (arXiv:2112.09141) and subsequent papers for the methods and calibration. GALSF_ISMDUSTCHEM_GRAINSIZEEVO can be additionally enabled to track grain size evolution with logarithmically-spaced size bins (set the value to the number of bins, e.g. 16), enabling coagulation, shattering, accretion, and sputtering as a function of grain size; this requires GALSF_ISMDUSTCHEM_MODEL=1 or (1+2) only, together with GALSF_ISMDUSTCHEM_SILICATE_COMPOSITION. Cite Choban et al. for the grain size evolution methods. GALSF_ISMDUSTCHEM_SILICATE_COMPOSITION sets the chemical composition assumed for the silicate dust, which in turn changes the production, growth and destruction rates of the silicate species and the maximum depletions of Mg, Fe, Si and O permitted in the gas phase; the value is a sum of bitflags: 1=an olivine-pyroxene mix, which is the default, 2=add two extra oxygen atoms to better match the observed oxygen depletions, 4=add one extra iron atom to better match the iron depletions, 8=remove all iron from the silicates, which is meant to be used together with the metallic iron species. This is read as a plain integer rather than tested bit by bit, so it is always defined whenever the dust model is active, defaulting to 1 if it is not set explicitly; (1+2+8) is a common choice. It is required by the grain size evolution above.

Sink-Particle Dynamics (IMF, Star formation, Planets, Compact Objects)

This set of flags controls sink particle dynamics for simulations with mass-resolved sink particles intended to represent point-like, accreting subsystems. This is most commonly used for simulations of individual star formation or formation of populations of stars (e.g. simulations modeling the stellar IMF), but can also be used for representing formation or dynamics of other un-resolved accreting systems (planets, compact objects such as black holes and neutron stars, etc.) as well as N-body dynamics of star clusters with individual stars. This uses the same fundamental algorithms as super-massive black holes for galaxy simulations, which are described in a separate section below, but in those cases, a different set of feedback assumptions applies. The evolution and feedback modules here are generally based on the assumption that the sinks represent proto-stars (or planets) and/or stars, but the general dynamics are indepednent of this assumption.

The core algorithms and underlying numerical framework for all of these physics and modules is now public. The specific stellar evolution tracks and modifications to the public modules made for the STARFORGE project remain in development: please contact their developers (Mike Grudic and David Guszejnov) before using the full STARFORGE suite. 

############################################################################################################################
## ----------------------------------------------------------------------------------------------------
# --------------- Star+Planet+Compact Object Formation (Sink Particle + Explicit/Keplerian N-Body Dynamics)
# -------------------- (unlike GALSF options, these sinks are individual accretors, not populations). Much in common with black hole-centric modules below.
# -------------------- These are in active development with Mike Grudic, David Guszejnov, and PFH: please contact them before using these modules, though basic modules may be ok (there is a lot of development happening here)
## ----------------------------------------------------------------------------------------------------
#SINGLE_STAR_SINK_DYNAMICS      # top-level switch to enable any other modules in this section
############################################################################################################################

SINGLE_STAR_SINK_DYNAMICS: Top-level switch. Must be on for any of the other modules within this block to work. These are essentially different ‘skins’ or options for the sink particle modules, which are described below, so make sure to read both sections if you are planning on using these. More details of the generic sink particle modules (including e.g. accretion and dynamics options) are found in the notes_blackholes file (here). Additional details that are unique to the sink modules in this section will be described in the (currently in-progress) notes_singlestar file (here), until the methods paper is published.

N-Body Dynamics, Integration, Regularization

Gravity is solved in sink-particle simulations using the same fundamental methods as any other simulations, described above. True point particles can be represented in tree-gravity, simply taking the limit of vanishingly small softening. Depending on the problem, various choices for sink gravity softening and tree versus particle-mesh grids may be optimal. However, a common issue which occurs in simulations with explicit sinks intended to represent unresolved massive point-like particles is the formation of hard binaries or multiples which evolve on much faster timescales as compared to the rest of the simulation. For example, in simulations of star formation with individual sink stars, close binaries will form, which (especially if eccentric) will have close passages requiring very small timesteps to accurately evolve, or else the orbits will tend to degrade over time (if e.g. highly-softened). There are a range of options for dealing with this. 

############################################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- time integration, regularization, and explicit small-N-body dynamical treatments (for e.g. hard binaries, etc)
## ----------------------------------------------------------------------------------------------------
#SINGLE_STAR_TIMESTEPPING=1     # use additional timestep criteria to ensure resolved binaries/multiples dont dissolve in close encounters. 0=most conservative. 1=super-timestep hard binaries by operator-splitting the binary orbit. 2=more aggressive super-timestep. cite Grudic et al., arXiv:2010.11254, for the methods here.
#HERMITE_INTEGRATION=32         # Instead of the usual 2nd order DKD Leapfrog timestep, do 4th order Hermite integration for particles matching the bitflag. Allows longer timesteps and higher accuracy collisional dynamics. cite Grudic et al., arXiv:2010.11254, for the methods here.
#DISABLE_HERMITE_INTEGRATION    # negate the HERMITE_INTEGRATION that SINGLE_STAR_STARFORGE_DEFAULTS would otherwise switch on, leaving sinks on plain KDK leapfrog. Useful as a convergence control, leapfrog being 2nd order.
#SINGLE_STAR_RT_DEFAULTS        # the RT transport settings the STARFORGE radiative modules share (M1 solver, comoving frame, reduced speed of light, sink photon injection), without the radiative-feedback band set. SINGLE_STAR_FB_RAD switches this on for you.
#JET_DIRECTION_FIXED_Z          # TESTING ONLY: launch spawned jets along +/-z instead of the accreted angular-momentum axis, so a launch geometry can be checked against a known direction. Not physical for production.
#WIND_MDOT=1e-7                 # TESTING ONLY: hard-code the stellar wind mass-loss rate [Msun/yr] instead of taking it from the stellar evolution model, for controlled wind tests.
#WIND_LUMINOSITY=1e36           # TESTING ONLY: hard-code the wind luminosity [erg/s]; with WIND_MDOT this sets the wind speed via L_w=(1/2) Mdot v_w^2.
#IO_HERMITE_SYNC                # write HermiteSyncCoordinates/HermiteSyncVelocities: a mutually consistent position-velocity pair at the output time, unlike the ordinary drift-time positions against kick-time velocities. Needed for orbital elements, vis-viva or kinetic energy from a snapshot.
## ----------------------------------------------------------------------------------------------------
############################################################################################################################

Normally, the default error tolerances in GIZMO’s tree-gravity solver will allow relatively large degradation of tight binary orbits, because it is optimized for many-body dynamics with softened gravity (where individual resolution elements are not point-masses). If high accuracy in close few-body encounters is desired, a few flags will ensure this: several of these are rolled together into the convenience flag GRAVITY_ACCURATE_FEWBODY_INTEGRATION, which includes GRAVITY_HYBRID_OPENING_CRIT (uses both a Barnes-Hut and relative acceleration tree opening criterion, to be more conservative), LONG_INTEGER_TIME and STOP_WHEN_BELOW_MINTIMESTEP (to deal with deeper timestep hierarchies), and TIDAL_TIMESTEP_CRITERION to timestep carefully through close passages. However, this may drop the timesteps to very small values. To deal with this, the code modules here allow for some major optimizations, developed by Mike Grudic.

SINGLE_STAR_TIMESTEPPING: If enabled, the code will use additional timestep criteria to ensure resolved binaries/multiples dont dissolve in close encounters. Basically, this module operator-splits long-range gravitational forces and short-range forces from hard binaries and other multiples. Using regularization techniques from the N-body dynamics community, this identifies hard binaries in the gravity tree, then computes the long-range TreePM forces on the center-of-mass location of the binary. The binary forces are then solved in an operator-split manner: we explicitly solve for the update assuming the external force is frozen (including the tidal tensor) across the timestep for the ‘long timestep’ of the binary, while the orbit dynamically evolves, which allows one to take a much longer timestep than if one simply used the normal tree-gravity step evolve the binary. If this is on, it must be set to an integer value to determine how aggressive the timestepping actually is. If set =0, the most conservative mode is used (the operator splitting occurs, but a conservative timestep needed to directly integrate the binary is still adopted) – so this is more accurate, but does not accelerate the timestepping. If set =1, the binaries are ‘super-timestepped’ as described above. If set =2, the super-timestepping is more aggressive (the most aggressive it can be before becoming numerically unstable). Cite Grudic et al., arXiv:2010.11254, for the methods here.

HERMITE_INTEGRATION: Instead of the usual 2nd order drift-kick-drift leapfrog timestep used in GIZMO, enabling this flag will do 4th order Hermite integration for any particles matching the bitflag. Note that the Hermite integration is designed (and guaranteed to give nice, Hermitian behavior and benefits) only for collisionless particles evolving under gravity only, so only that part of the position, velocity, and accelerations will be integrated according to this (in other words, this applies only to the N-body parts of the code, not to e.g. integrations of hydrodynamics or other physics). Set the value equal to a bitflag defining the particles for which the integration will be used. This will automatically enable tidal tensor calculations and timesteps based on the tidal timestep: but together with the SINGLE_STAR_TIMESTEPPING flag above the real benefits are most apparent, as this allows one to take much larger (nearly order-of-magnitude) timesteps for hard binaries and multiples while maintaining ideal conservation and other desirable properties of a Hermitian integrator. Cite Grudic et al., arXiv:2010.11254, for the methods here.

DISABLE_HERMITE_INTEGRATION: Negates the HERMITE_INTEGRATION which SINGLE_STAR_STARFORGE_DEFAULTS would otherwise switch on, leaving the sinks on the ordinary KDK leapfrog. This is useful as a convergence control: leapfrog is 2nd order where the Hermite scheme is 4th, so running the same problem both ways separates the errors which come from the integrator from those which do not.

SINGLE_STAR_RT_DEFAULTS: Enables the radiation-transport settings which the STARFORGE radiative modules have in common – the M1 solver, the comoving frame, the reduced speed of light, and photon injection from sinks – without the radiative-feedback band set which would ordinarily accompany them. SINGLE_STAR_FB_RAD switches this on for you, so you only need to set it directly if you want that transport configuration without the feedback bands.

JET_DIRECTION_FIXED_Z: For testing only. Launches spawned jets along \(\pm z\) rather than along the accreted angular-momentum axis, so that the launch geometry can be checked against a known direction. This is not physical and should not be enabled for science runs.

WIND_MDOT: For testing only. Hard-codes the stellar wind mass-loss rate, in \({\rm M_{\odot}\,yr^{-1}}\), instead of taking it from the stellar evolution model, so that wind injection can be exercised at a controlled and known rate.

WIND_LUMINOSITY: For testing only. Hard-codes the wind luminosity in erg/s; together with WIND_MDOT this fixes the wind velocity through \(L_{w}=(1/2)\dot{M}_{w}v_{w}^{2}\).

IO_HERMITE_SYNC: Writes HermiteSyncCoordinates and HermiteSyncVelocities into snapshots: a mutually consistent position-velocity pair evaluated at the output time. The ordinary snapshot fields are not this, since the positions are at the drift time while the velocities are at the kick time. That mismatch is harmless for most purposes, but it is wrong if you want to compute orbital elements, apply vis-viva, or take a kinetic energy directly from a snapshot, so enable this if you need any of those.

Sink Creation and Accretion

These modules allow for sinks to be created and grow (as well as merge) ‘on-the-fly’ in the simulations. 

############################################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- sink creation and accretion/growth/merger modules
## ----------------------------------------------------------------------------------------------------
#SINGLE_STAR_SINK_FORMATION=(0+1+2+4+8+16+32+64) # form new sinks on the fly, criteria from bitflag: 0=density threshold, 1=virial criterion, 2=convergent flow, 4=local extremum, 8=no sink in kernel, 16=not falling into sink, 32=hill (tidal) criterion, 64=Jeans criterion, 128=converging flow along all principle axes, 256=self-shielding/molecular, 512=multi-free-fall (smooth dependence on virial). cite Grudic et al., arXiv:2010.11254, for the methods here.
#SINGLE_STAR_ACCRETION=7        # sink accretion [details in BH info below]: 0-10: use SINK_GRAVACCRETION=X, 11: SINK_GRAVCAPTURE_GAS, 12: SINK_GRAVCAPTURE_GAS modified with Bate-style FIXEDSINKRADIUS.  cite Grudic et al., arXiv:2010.11254, for the methods here.
## ----------------------------------------------------------------------------------------------------
############################################################################################################################

SINGLE_STAR_SINK_FORMATION: With this enabled, sink particles can be created ‘on the fly’ as opposed to simply in the ICs. You set the value to a bitflag which allows users to specify any arbitrary combination of criteria for star formation (provided they are physically compatible): the bitflag is set to the sum of the values corresponding to each criterion you want to enable, including: (0) A density threshold (set by the CritPhysDensity parameter in the params file). Note because the value is 0 here, this is effectively always on, but you can essentially disable it by simply setting CritPhysDensity very low or zero. (1) A local virial criterion (self-gravitating at the resolution scale including kinetic, thermal, and magnetic energy (see the description above for details of what this entails and for the proper references if you use it). (2) Requires the gas be in a convergent flow (\(\nabla\cdot{\bf v}<0\)). (4) Require the gas cell sit at a local density and potential extremum (evaluated within kernel). (8) Require that there are no other sinks inside three times the gas kernel/neighbor radius, and the gas is not a valid ‘interacting neighbor’ of any other sink particle. (16) Require the timescale for the gas element to free-fall onto (using a 2-body orbit) or intercept (\(\sim 1/(\Delta{\bf v}\cdot \Delta {\bf r})\)) the nearest sink is longer than the local dynamical collapse time. Cite Grudic et al. arXiv:2010.11254 for the algorithm. (32) Enforce a Hill-type criterion that the local tidal tensor is negative definite (forces are converging along all three principle axes) and that the gas cell lives inside its own Hill sphere. This is a much stronger and more rigorous (Galilean-invariant) version of a local maximum-type criterion. Cite Grudic et al. arXiv:2010.11254 for the algorithm. (64) Require that the thermal+magnetic Jeans mass be less than the minimum of 1e4 solar masses or 100x the gas cell mass (but no smaller than 1e–3 solar masses). Cite Grudic et al. arXiv:2010.11254. (128) Require that the flow be converging along all three principle axes by diagonalizing the velocity shear tensor. This is a much stricter version of the converging-flow criterion, though not necessarily always more physical. (256) Require that the gas be self-shielding (molecular) and scale the star formation rate to the molecular fraction in the gas cell. (512) Scale the star formation efficiency per free-fall time not to a constant (the default behavior), but to a multi-free-fall model (this is generally only relevant in galaxy-scale simulations; see the description in GALSF_SFR_CRITERION above). (1024) Adds a purely-numerical ‘catch’ where the more complicated kinematic criteria are ignored if the gas kernel size approaches the minimum force softening, where the forces become non-Keplerian and so some of the criteria above (like the virial criteria) are fundamentally ill-defined. (2048) Uses a time-averaged version of the virial criterion (see description above again). If all of the chosen criteria are met, a gas element is immediately converted into a sink. Each of these options involves papers that should be cited: please see the code and documentation for sink modules to cite the appropriate paper[s]. Cite Grudic et al., arXiv:2010.11254, for the methods here.

SINGLE_STAR_ACCRETION: Enables accretion onto sinks after they form. The value chooses the model, but all are described in the sinks documentation (please read that for details of how the different modules work in detail). Values 0–10 choose ‘continuous accretion rate’ models, where the accretion rate onto the sink is continuously evaluated from the gas properties of the neighbors through which the sink moves. Within this, values 0–10 specifically are identical to the corresponding value of SINK_GRAVACCRETION: 0=estimator from Hopkins and Quataert 2010, for accretion in a gravitationally-unstable disk, 1=revision to 0 from Angles-Alcazar et al 2011, 2=fixed accretion per free-fall time from accretion radius, 3=gravito-turbulent scaling from Gammie et al., 4=fixed accretion per free fall within radius where sink dominates potential, 5=hybrid Bondi plus self-gravitating disk model, 6=fixed inflow per free-fall in collisionless Hernquist sphere, 7=hybrid model interpolating between Shu-type isothermal sphere accretion and Bondi-Hoyle, 8=Hubber-type disk-free-fall interpolation, 9=pure Bondi-Hoyle (ignoring the possibility of gas angular momentum or cooling), 10=pure Bondi (ignoring gas velocity). Setting this flag =11 or =12 uses the SINK_GRAVCAPTURE_GAS flag, where accretion is not continuous but discrete, swallowing gas elements which are bound within the neighbor radius and meet some other basic criteria. =11 sets the default behavior for this flag, =12 sets it to behave like the Bate-style sinks (with a fixed, instead of adaptive, accretion radius, and an angular momentum criterion for accretion). Each of these options involves papers that should be cited: please see the code and documentation for sink modules to cite the appropriate paper[s]. Cite Grudic et al., arXiv:2010.11254, for the methods here.

Star and Planet Formation Physics

The flags above make no assumptions about what, physically, the sinks are intended to represent. This block of flags represents flags which are designed specifically for simulations of star and/or planet formation, where the sinks represent accreting objects in the mass range anywhere from super-earths through hyper-massive quasi-stars, but are non-relativistic and obey some basic equations of stellar/planetary structure and evolution. 

############################################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- star (+planet) formation-specific modules (feedback, jets, radiation, protostellar evolution, etc)
## ----------------------------------------------------------------------------------------------------
#SINGLE_STAR_STARFORGE_PROTOSTELLAR_EVOLUTION=1 # sinks are assumed to be proto-stars and follow protostellar evolution tracks as they accrete to evolve radii+luminosities, determines proto-stellar feedback properties. 1=simple model [PFH], 2=fancy model [DG+MG]. Cite Grudic et al., arXiv:2010.11254, for the methods here.
#SINGLE_STAR_FB_JETS            # kinematic jets from sinks: outflow rate+velocity set by Sink_f_accretion+Sink_v_outflow. cite Angles-Alcazar et al., 2017, MNRAS, 464, 2840 (for algorithm, developed for black hole jets), and cite algorithm developed by KY Su. cite Su et al, arXiv:2102.02206, and Grudic et al., arXiv:2010.11254, for the methods here.
#SINGLE_STAR_FB_WINDS           # enable continuous main-sequence mechanical feedback from single stellar sources accounting for OB/AGB/WR winds. with STARFORGE parent flag[s] enabled, this will following STARFORGE methods (Grudic+ arXiv:2010.11254). Otherwise, this will follow a simpler Castor, Abbot, & Klein scaling, for type=4 particles representing single stars, using the standard GALSF_FB_MECHANICAL algorithms in code, for which you should cite Hopkins et al. 2018MNRAS.477.1578H
#SINGLE_STAR_WIND_MODE=1        # pin the jets/winds discrete-spawn channel (1=winds spawn, 2=winds injected continuously) instead of letting the momentum-rate comparison choose, to exercise one injection path on its own
#SINK_SPAWN_MERGE_WHEN_AMBIENT  # retire a spawned jet/wind cell only once it is subsonic and thermally equilibrated with the non-spawned gas around it, not merely subsonic wrt its merge target. auto-set with the STARFORGE protostellar model
#SINK_SPAWN_MERGE_MAX_THERMAL_CONTRAST=(1.0)  # max |ln(u_cell/u_ambient)| for a spawned cell to count as having joined the ambient medium
#SINK_SPAWN_MERGE_ANY_NEIGHBOR  # opt out of SINK_SPAWN_MERGE_WHEN_AMBIENT, restoring the retire-when-subsonic-wrt-target behaviour
#MERGE_SPLIT_LIMIT_KINETIC_DISSIPATION  # never retire a spawned cell into a target if that would thermalize the majority of the energy involved, which is an unresolved shock. auto-set with the STARFORGE protostellar model
#MERGE_SPLIT_MAX_KINETIC_DISSIPATION_FRACTION=(0.5)  # max KE_com/(KE_com+E_light) allowed by the above; lower is stricter (~0.34 is Mach 1 on the lighter cell)
#MERGE_SPLIT_ALLOW_KINETIC_DISSIPATION  # opt out of MERGE_SPLIT_LIMIT_KINETIC_DISSIPATION
#SINGLE_STAR_FB_SNE             # enable supernovae from single stellar sources at end of main-sequence lifetime. with STARFORGE flag[s] enabled this will use particle spawning in shells following STARFORGE methods (Grudic+ arXiv:2010.11254). Otherwise, this will act uniformly and in a single timestep at the end of the stellar main-sequence lifetime for type=4 particles representing single stars, using the standard GALSF_FB_MECHANICAL algorithms in code, for which you should cite Hopkins et al. 2018MNRAS.477.1578H
#SINGLE_STAR_FB_RAD             # enable radiative feedback from stars, hooking into the standard radiation hydrodynamics algorithms. you need to determine how the effective temperatures of the stars scale, which will be used to determine their input fluxes into the different explicitly-evolved bands.
## ----------------------------------------------------------------------------------------------------
############################################################################################################################

Again a reminder some of the STARFORGE-specific modules are under-development, though they are all described in the methods paper Grudic et al., arXiv:2010.11254. This should be considered to update and replace the older notes_singlestar file (here). If you are interested in co-development, please contact Mike Grudic and David Guszejnov.

SINGLE_STAR_STARFORGE_PROTOSTELLAR_EVOLUTION: If this flag is enabled, sinks are assumed to initially be proto-stars (or proto-planets if low enough mass). They follow protostellar evolution tracks as they accrete: this allows you to explicitly follow e.g. the radius of the collapsing object along the Henyey and Hayashi tracks as it both contracts and accretes. Feedback properties (luminosities, jet properties, etc) are calculated appropriately for the stage of evolution and present-state properties of the object. Cite Grudic et al., arXiv:2010.11254, for the methods here: you are encouraged to cite also the other core initial papers of the STARFORGE suite: arXiv:2010.11254, arXiv:2010.11249, arXiv:2201.01781, arXiv:2201.00882

SINGLE_STAR_FB_RT_HEATING: Adds feedback from accreting proto-stars in the form of heating via their accretion luminosity. By default this performs a LEBRON-method radiation hydrodynamic simulation, with the stellar luminosity coming out in the IR. If the radiation hydro module is changed, whatever radiation hydro method is chosen will be used. The accretion luminosity scales with the accretion rate given the radiative efficiency, set by the run-time parameter SinkRadiativeEfficiency, unless protostellar evolution is enabled, in which case the appropriate proto-stellar evolution tracks are used to determine the accretion and contraction luminosities (both are then used for feedback). See description of the radiative feedback modules in the sink notes for more details of the numerical methods. Importantly, if you are using the public code, then by enabling explicit radiation-hydrodynamics (also in the public code), this is not needed, and the user can treat full radiative feedback in the public code without needing any of the in-development modules here.

SINGLE_STAR_FB_JETS: Adds feedback from proto-stellar jets. This acts as as kinematic outflows with a specified outflow rate set in the same manner as black hole-driven accretion-disk outflows. This leverages the SINK_WIND_SPAWN module, see the description in the sink particle notes for the details of the numerical methods. The run-time parameters Sink_accreted_fraction or Sink_f_accretion (both set the same variable, your choice of the convention to use), Sink_outflow_temperature or Cell_Spawn_Temperature, and Cell_Spawn_Mass_ratio, are used, as is standard with this module (see the parameterfile notes below for the sink/sink-particle modules) to determine the mechanical and thermal energy-loadings (as well as e.g. optional magnetic or cosmic-ray loading) for the jets. The jet launch speed is not set by Sink_outflow_velocity: it is Sink_outflow_jetlaunchvelscaling (or Sink_f_launch_v) times the Keplerian speed at the protostellar radius. The jets, by default, are collimated with a very small opening angle at launch (they almost immediately will expand as they propagate), along the direction of the sink accreted specific angular momentum. Cite Su et al, arXiv:2102.02206, and Grudic et al., arXiv:2010.11254, for the methods here.

SINGLE_STAR_FB_WINDS: Adds continuous main-sequence mechanical feedback from single stellar sources accounting for OB/AGB/WR winds. This adds the parameter Cell_Spawn_Mass_ratio_MS which can be set for the cell-spawning for this mode of feedback analogous to Cell_Spawn_Mass_ratio, but allowing you to specify these separately. This has two possible algorithmic channels it can adopt, depending on other flags that are set. With STARFORGE parent flag[s] enabled, this will following STARFORGE methods (Grudic+ arXiv:2010.11254). Otherwise, this will follow a simpler Castor, Abbot, & Klein scaling, for type=4 particles representing single stars, using the standard GALSF_FB_MECHANICAL algorithms in code, for which you should cite Hopkins et al. 2018MNRAS.477.1578H. Users interested in modifying the relevant scalings or active particle types should modify the code appropriately in the stellar_evolution.c file.

SINGLE_STAR_WIND_MODE: Pins the channel through which jets and winds are injected, instead of letting the code select it by comparing momentum rates. Set =1 to force winds to spawn as discrete cells, or =2 to force them to be injected continuously. This exists to exercise one injection path on its own; the automatic comparison is what you want otherwise.

SINK_SPAWN_MERGE_WHEN_AMBIENT: Retires a spawned jet or wind cell only once it is both subsonic and thermally equilibrated with the non-spawned gas around it, rather than merely subsonic with respect to the cell it is being merged into. The distinction matters because a spawned cell can be subsonic relative to its merge target while remaining far hotter than the medium, in which case retiring it deposits that thermal energy somewhere it does not belong. This is set automatically with the STARFORGE protostellar model.

SINK_SPAWN_MERGE_MAX_THERMAL_CONTRAST: Sets how nearly a spawned cell must match the ambient medium before the criterion above will retire it, as the largest allowed \(|\ln{(u_{\rm cell}/u_{\rm ambient})}|\). The default is 1.0.

SINK_SPAWN_MERGE_ANY_NEIGHBOR: Opts out of SINK_SPAWN_MERGE_WHEN_AMBIENT, restoring the older behavior in which a spawned cell is retired as soon as it is subsonic with respect to its merge target, whatever its temperature relative to the surrounding gas.

MERGE_SPLIT_LIMIT_KINETIC_DISSIPATION: Refuses to retire a spawned cell into a target if doing so would thermalize the majority of the energy involved. Merging two cells carrying a large relative velocity converts their relative kinetic energy into heat in a single step, which is an unresolved shock rather than a physical one, and this prevents it. Set automatically with the STARFORGE protostellar model.

MERGE_SPLIT_MAX_KINETIC_DISSIPATION_FRACTION: Sets the fraction permitted by the criterion above, as \(KE_{\rm com}/(KE_{\rm com}+E_{\rm light})\). Lower values are stricter; roughly 0.34 corresponds to Mach 1 on the lighter of the two cells. The default is 0.5.

MERGE_SPLIT_ALLOW_KINETIC_DISSIPATION: Opts out of MERGE_SPLIT_LIMIT_KINETIC_DISSIPATION, permitting merges however much kinetic energy they would thermalize.

SINGLE_STAR_FB_SNE: Adds supernovae from single stellar sources at end of main-sequence lifetime. Again this has two possible behaviors, depending on the other flags that are set. With STARFORGE flag[s] enabled this will use particle spawning in shells following STARFORGE methods (Grudic+ arXiv:2010.11254). Otherwise, this will act uniformly and in a single timestep at the end of the stellar main-sequence lifetime for type=4 particles representing single stars, using the standard GALSF_FB_MECHANICAL algorithms in code, for which you should cite Hopkins et al. 2018MNRAS.477.1578H. Users interested in modifying the relevant scalings or active particle types should modify the code appropriately in the stellar_evolution.c file.

SINGLE_STAR_FB_RAD: Adds radiative feedback from stars, hooking into the standard radiation hydrodynamics algorithms. you need to determine how the effective temperatures of the stars scale, which will be used to determine their input fluxes into the different explicitly-evolved bands. The user can modify these in the function stellar_lum_in_band in rt_utilities. By default it will assume a simple main-sequence scaling, unless explicit stellar evolution tracks are enabled (as in e.g. the STARFORGE models). Cite Grudic+ arXiv:2010.11254 for use.

Note that the flags EOS_SUBSTELLAR_ISM and/or EOS_GMC_BAROTROPIC are often especially relevant here – those are described above in the EOS section, but search for them to find details.

Optional and De-Bugging Modules

There are a variety of flags for additional special behaviors. Here are several most commonly used for the types of simulations above. However users should also carefully read the section on super-massive sink particles below, as almost all of those flags are cross-compatible here: in fact the modules here and there are often the same, simply with different operator logic for the sake of user convenience.

############################################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- optional and de-bugging modules (intended for specific behaviors)
## ----------------------------------------------------------------------------------------------------
#SINK_RETURN_ANGMOM_TO_GAS        # BH/sink particles return accreted angular momentum to surrounding gas (per Hubber+13) to represent AM transfer (loss in accreting material)
#SINGLE_STAR_FIND_BINARIES      # manually enable identification of close binaries (normally enabled automatically if actually used for e.g. hermite timestepping). cite Grudic et al. arXiv:2010.11254
#SINGLE_STAR_FB_LOCAL_RP        # approximate local radiation pressure from single-star sources, using the same LEBRON-type approximation as in FIRE - useage follows the FIRE collaboration policies
#SINGLE_STAR_FB_RT_HEATING      # proto-stellar heating: luminosity determined by SinkRadiativeEfficiency (typical ~5e-7). This particular module used without radiation-hydrodynamics uses FIRE modules, so citations follow those. But by enabling explicit radiation-hydrodynamics, this is not needed, and the user can treat full radiative feedback in the public code.
#SINGLE_STAR_FB_SNE_N_EJECTA_QUADRANT=2  # determines the maximum number of ejecta particles spawned per timestep in the supernova shell approximation for spawning. only needs to be modified for testing purposes.
############################################################################################################################

SINK_RETURN_ANGMOM_TO_GAS: In principle, accreted gas should be losing angular momentum. While in general this will include largely un-resolved processes ‘internal’ to the sink, and even the angular momentum lost back to the ambient medium outside the sink will be in the form of radiation, magnetic fields, jets, winds, and torques between neighboring elements (including other sinks in the case of binaries, etc), it is nonetheless desired in some cases to mock up the effect as if all of the angular momentum of material captured by the sink were instantly transferred to the immediately surrounding gas. This flag enables ‘angular momentum return’ in this fashion, following the prescription in Hubber et al. 2013 (where we follow the total accreted angular momentum and inject angular momentum continuously to neighbor gas elements, at a rate proportional to the accretion rate onto the sink).

SINGLE_STAR_FIND_BINARIES: Manually enable identification of close binaries on-the-fly in the code (normally enabled automatically if actually used for e.g. hermite timestepping). cite Grudic et al. arXiv:2010.11254

SINGLE_STAR_FB_LOCAL_RP: Adds an approximate local radiation pressure from single-star sources, using the same LEBRON-type approximation as in FIRE - useage follows the FIRE collaboration policies. But by enabling explicit radiation-hydrodynamics, this is not needed, and the user can treat full radiative feedback in the public code.

SINGLE_STAR_FB_RT_HEATING: Adds proto-stellar heating with a simple non-explicit-RHD approximation, where the luminosity determined by SinkRadiativeEfficiency (typical might be 5e–7 or so). This particular module used without radiation-hydrodynamics uses FIRE modules, so citations follow those. But by enabling explicit radiation-hydrodynamics, this is not needed, and the user can treat full radiative feedback in the public code.

SINGLE_STAR_FB_SNE_N_EJECTA_QUADRANT: This manually determines the maximum number of ejecta particles spawned per timestep in the supernova shell approximation for spawning. only needs to be modified for testing purposes. Default =2.

SuperMassive Black Hole Options

This set of flags controls the physics of black holes (or really, compact accreting objects/sink particles of any mass). This includes their formation/seeding (on-the-fly in simulations), their accretion, growth, dynamics in the code, and feedback (radiative and mechanical). All of these modules are under some degree of development; some as part of existing PhD theses or other projects such as FIRE. Before using a module still in development, consult PFH and the relevant authors of the code. Additional details of the modules are described in the notes_blackholes file (here), until the methods paper is published.

####################################################################################################
# ---------------- sink particles (Sink-Particles with Accretion and Feedback)
####################################################################################################
#SINK_PARTICLES                    # top-level switch to enable sink particle physics (for BHs, stars, etc)
####################################################################################################

SINK_PARTICLES: Top-level switch for BH physics. Must be on for any BH-related physics to work. BHs will always be particle type 5.

BH particles, like star particles, carry a number of additional variables with them that are used by the code and saved in the snapshot outputs. See the sections below about snapshots for details on those outputs. For example, a BH particle has a “BH mass” (mass of the assumed BH, which can be arbitrarily small) and a “particle/dynamical mass” (mass of the actual particle in the code, which is used for the gravity equations, etc). The latter can, in principle, be much larger than the BH mass (so the BH is ‘sub-grid’ in mass, in these cases).

Seeding and BH-Creation

BH particles can, of course, be placed in the initial conditions. But if you want them to be spontaneously generated over the course of the simulation, one of these options must be set to determine the rule by which BHs are “spawned.”

All seed models require certain parameters in the parameterfile: (1) SeedSinkMass = the actual seed BH mass, in code units. (2) SeedSinkMinRedshift = the minimum redshift where seeding is allowed – at redshifts below this, no seeds will be created. (3) SeedSinkMassSigma = logarithmic dispersion in BH mass, in dex – BHs will be spawned with a random mass drawn from a lognormal with this width.

####################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- seeding / BH-particle spawning
## ----------------------------------------------------------------------------------------------------
#SINK_SEED_FROM_FOF=0             # use FOF on-the-fly to seed BHs in massive FOF halos; =0 uses DM groups, =1 uses stellar [type=4] groups; requires FOF with linking type including relevant particles (cite Angles-Alcazar et al., MNRAS, 2017, arXiv:1707.03832)
#SINK_SEED_FROM_LOCALGAS          # BHs seeded on-the-fly from dense, low-metallicity gas (no FOF), like star formation; criteria define-able in modular fashion in sfr_eff.c (function return_probability_of_this_forming_sink_from_seed_model). cite Grudic et al. (arXiv:1612.05635) and Lamberts et al. (MNRAS, 2016, 463, L31). Requires GALSF and METALS
##SINK_INCREASE_DYNAMIC_MASS=100  # increase the particle dynamical mass by this factor at the time of BH seeding
####################################################################################################

SINK_SEED_FROM_FOF: The code will run an on-the-fly friends-of-friends group finder, and when a group exceeds a critical mass and does not already contain a BH, one will be created from one of the particles which is a member of that group (the particle at the potential minimum of the group). This module requires FOF is active (obviously), with a linking type that includes the relevant particle types being used for the actual seeding. This flag can be set to one of two values at present. If “0”, then the dark-matter groups are used to identify the FOF parent halos, and the BH particle is spawned from the gas particle at the potential minimum (among gas particles in the halo). If “1”, then the groups used are stars (e.g. stellar galaxies as opposed to halos), and the BH is spawned from the star particle at the potential minimum. In the parameterfile, several parameters must be set: (1) MinFoFMassForNewSeed = minimum FOF group mass (in code units) for seeding. Note if the FOF parent group is DM, this is the DM halo mass, if the FOF parent group is stars, this is the stellar mass. (2) TimeBetOnTheFlyFoF = time (in code units, so scale factor for cosmological runs) between forcing FOF searches for BH seeding. Users of this module should cite Angles-Alcazar et al., MNRAS, 2017, arXiv:1707.03832.

SINK_SEED_FROM_LOCALGAS: Activate “on-the-fly” seeding of BHs based on local gas properties (instead of FOF group-finding), more appropriate for e.g. highly-resolved (zoom-in or sub-galactic) simulations. When a gas particle is flagged as being “turned into” a star particle, then we assign it some probability of instead turning into a “seed” BH, where the probability is specified by an arbitrary user function (return_probability_of_this_forming_sink_from_seed_model in the file stellar_evolution.c). The coded example there has a probability that increases strongly in higher-surface density, lower-metallicity gas (default values in the code assume the probability increases significantly above a surface density \(>1\,{\rm g\,cm^{-2}}\) and below metallicity \(<0.01\) solar), motivated by the simulations and stellar evolution calculations in Grudic et al. (arXiv:1612.05635) and Lamberts et al. (MNRAS, 2016, 463, L31); please cite these if you use something like these default values. But users are encouraged to code up their own seeding prescriptions here. Note that the default version here requires GALSF and METALS are active.

SINK_INCREASE_DYNAMIC_MASS: Increases the particle dynamical mass (the total particle mass which enters the gravity equations, as distinct from the ‘sink particle’ mass) by the specified factor (its default value will be the particle mass of whatever is being ‘converted’ into the BH), when a BH seed is assigned. Used in some circumstances to address the BH dynamics questions below, or as a sub-grid model for BH formation in massive star clusters.

BH Dynamics (Dynamical Friction, Drag)

In cosmological simulations with poor resolution, it is commonly the case that the BHs have mass smaller than the mass of typical neighbor particles, even though the physical BH mass should be large compared to the mass of the true individual stars (let alone DM particles) it is moving through. This has an important consequence: rather than sinking to the galaxy center via dynamical friction, the BH will not be ‘special’ (or can even get kicked around from numerical N-body noise). These options address this with some fixes of varying realism and efficacy.

####################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- dynamics (when BH mass is not >> other particle masses, it will artificially get kicked and not sink via dynamical friction; these 're-anchor' the BH for low-res sims)
## ----------------------------------------------------------------------------------------------------
#SINK_DYNFRICTION_FROMTREE         # compute dynamical friction forces on BH following the discrete DF estimator in Linhao Ma et al., arXiv:2101.02727. This is a more flexible, general, and less noisy and more accurate version of the traditional Chandrasekhar dynamical friction formula. Cite L Ma et al. 2021 and 2022 if used, and contact author L. Ma for applications as testing is still ongoing.
#SINK_REPOSITION_ON_POTMIN         # reposition sink particle on potential minimum (requires EVALPOTENTIAL). moves smoothly with damped velocity to most-bound particle in kernel. Cite Wellons et al. arXiv:2203.06201
####################################################################################################

SINK_DYNFRICTION_FROMTREE: Compute dynamical friction forces on BH following the discrete dynamical friction estimator in Linhao Ma et al., arXiv:2101.02727 and L. Ma et al., arXiv:2208.12275. This is a more flexible, general, and less noisy and more accurate version of the traditional Chandrasekhar dynamical friction formula, and is recommended over that implementation for any simulations which wish to track the un-resolved dynamical friction forces in such a manner. Care is needed to avoid double-counting in the high-resolution limit. While in the public code, users are encouraged to contact Linhao Ma to discuss use, if desired, since some testing is still ongoing for certain applications. Please cite both Ma et al. papers if using this flag.

SINK_REPOSITION_ON_POTMIN: This is the most extreme option, and it simply anchors the BH particle to the potential minimum (it will re-position every timestep to the location of the potential minimum within the kernel). This uses the modern smoother version, obtained in older code versions by setting it equal to =2, which finds the most deeply-bound particle (not just potential minimum), and moves continuously with a velocity-damping term, all of which combines to give the most stable centering. The options are explained in more detail in the notes_blackholes file. The new option is detailed in Wellons+Angles-Alcazar+PFH et al., arXiv:2203.06201, which should be cited for its use.

BH Accretion Models

sink particles accrete, but this accretion is generally taking place at scales (the Schwartzschild radius) well below the resolution of the simulations. Therefore different sub-grid models are present to mimic accretion both into an un-resolved accretion disk around the BH, and from that onto the BH itself. All of the accretion models below should enable the SINK_SWALLOWGAS flag.

The accretion models involve several parameterfile settings. SinkAccretionFactor arbitrarily multiplies the calculated accretion rate by a constant factor (so its default value is =1). SinkEddingtonFactor caps the maximum accretion rate at this multiple of the Eddington limit (so Eddington-limited accretion sets this =1, un-limited accretion sets this to some huge value). SinkNgbFactor multiplies the standard kernel neighbor number – it is recommended this value be set to a few, since some of these estimators rely on things like bulge-disk decomposition inside the kernel (which requires a decent number of elements). SinkMaxAccretionRadius sets the maximum radius (in code units) out to which the BH will search for neighbors to achieve this neighbor number (so that it doesnt go indefinitely far, if there is little gas within the radius around the BH). SinkRadiativeEfficiency sets the radiative efficiency, important for both feedback and for determining what fraction of mass actually ends up in the BH versus is lost to radiation.

####################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- accretion models (modules for gas or other particle accretion)
## ----------------------------------------------------------------------------------------------------
#SINK_SWALLOWGAS                  # 'top-level switch' for accretion (should always be enabled if accretion is on). enables BH to actually eliminate gas particles and take their mass.
#SINK_ALPHADISK_ACCRETION         # gas accreted goes into a 'virtual' alpha-disk (mass reservoir), which then accretes onto the BH at the viscous rate (determining luminosity, etc). cite GIZMO methods (or PFH private communication)
#SINK_SUBGRIDBHVARIABILITY        # model variability below resolved dynamical time for BH (convolve accretion rate with a uniform power spectrum of fluctuations on timescales below the minimum resolved dynamical time). cite Hopkins & Quataert 2011, MNRAS, 415, 1027. Requires GALSF.
#SINK_GRAVCAPTURE_NONGAS          # accretion determined only by resolved gravitational capture by the BH, for non-gas particles (can be enabled with other accretion models for gas). cite Hopkins et al., 2016, MNRAS, 458, 816
## ----
#SINK_GRAVCAPTURE_GAS             # accretion determined only by resolved gravitational capture by the BH (for gas particles). cite Hopkins et al., 2016, MNRAS, 458, 816
#SINK_GRAVACCRETION=1             # family of gravitational/torque/angular-momentum-driven accretion models from Hopkins & Quataert (2011): cite Hopkins & Quataert 2011, MNRAS, 415, 1027 and Angles-Alcazar et al. 2017, MNRAS, 464, 2840. see `notes_blackholes` for details:
#                                 # [=0] evaluate at density kernel radius, [=1] evaluate at fixed physical radius, [=2] fixed efficiency per FF time at physical radius, [=3] gravito-turbulent scaling, [=4] fixed per FF at BH radius of influence, [=5] hybrid scaling (switch to Bondi if circularization radius small),
#                                 # [=6] modified bondi-hoyle/fixed accretion in sonic point for rho~r^-1 profile, [=7] shu+pressure+turbulence solution for isothermal sphere (self-similar isothermal sphere solution with these terms), [=8] hubber+13 estimator of local inflow (limited by 'external alpha-disk' and 'internal bondi' estimates)
#                                 # [=9] pure Bondi-Hoyle (ignore cooling/angular momentum), [=10] pure Bondi (dont use gas velocity with sound speed), [=11] variable-alpha tweak (Booth & Schaye 2009; requires GALSF)
#SINK_GRAVACCRETION_STELLARFBCORR # account for additional acceleration-dependent retention from stellar FB in Mdot. cite Hopkins et al., arXiv:2103.10444, for both the analytic derivation of these scalings and the numerical methods/implementation.
####################################################################################################

SINK_SWALLOWGAS: Allow the BH to actually consume gas particles. Required for any accretion model. Unless set otherwise by a specific compile flag, the accretion is stochastic, as in Springel et al. 2005 MNRAS 361 776 and Di Matteo et al. 2005 Nature 433 604 (see those papers for more details). The “sink” (or “black hole/star/central object”) mass is separately tracked from the particle mass (according to whatever accretion model). If the sink mass becomes larger than the particle mass (possible for some choices of parameters below), then nearby neighbor particles are immediately “swallowed” with some probability until the mass is balanced. This is done so that total particle mass in the simulation is properly conserved, without generating arbitrarily large mis-matches in the masses of different particle types.

SINK_ALPHADISK_ACCRETION: Enables a sub-grid model for an “alpha-disk” (a traditional Shakura & Sunyaev viscous disk) around the BH particle. If this is off, accretion into the BH will be instantaneous (whenever the large-scale accretion model determines there is accretion). If this is on, mass is accreted into the viscous disk, then accreted continuously out of that disk onto the BH at a rate following Shakura & Sunyaev (crudely this approaches Eddington when the mass in the disk approaches the pre-existing BH mass); both the BH mass and the viscous disk mass are separately and continuously followed. Note that although BH growth may be strictly limited at Eddington, accretion into the outer accretion disk is not. This smooths out the actual BH accretion rate over the viscous timescale. If you use this module, cite the GIZMO methods paper for now, since this has not been actually published.

SINK_SUBGRIDBHVARIABILITY: This enables a model for AGN variability on very small timescales, corresponding to internal variability in e.g. the un-resolved accretion disk. This is ‘smoothed over’ by finite resolution; however, we can crudely approximate it by including an explicit power-spectrum of fluctuations in dM/dt, integrated from frequencies of infinity down to 1/dt where dt is the simulation timestep (typically ~100 yr in the galaxy centers). We do this following Hopkins & Quataert 2011: assuming fluctuations in ln(dM/dt) follow a Gaussian random walk with equal power per logarithmic time interval from tmin (the orbital time at the innermost stable circular orbit for a non-rotating BH) to tmax (the dynamical time at the resolved BH kernel/accretion radius). If you use this module, cite Hopkins & Quataert 2011, MNRAS, 415, 1027 (the prescription is presented in their footnote 5). GALSF must also be enabled.

SINK_GRAVCAPTURE_GAS/SINK_GRAVCAPTURE_NONGAS: SINK_GRAVCAPTURE_GAS enables gas accretion via explicit resolution of direct gravitational capture of particles: if gas particles are located within the accretion radius (the minimum resolved softening around the BH), then this uses their position and relative velocity with respect to the BH particle we check whether (a) it is gravitationally bound to the BH, and (b) whether the apo-centric radius of the particle about the BH is also within the accretion radius. If both are true, we consider the particle “captured” and immediately add its mass to the BH accretion disk (if SINK_ALPHADISK_ACCRETION is enabled, otherwise it is added directly to the BH). Use this only if you want to force the code to resolve accretion into the viscous disk (i.e. this is for very high-res simulations). Otherwise de-activate this and use a “sub-grid” accretion rate estimator. SINK_GRAVCAPTURE_NONGAS does the same, for non-gas particle types (stars/dark matter; note that BH-BH mergers are always allowed, and handled separately from these flags). If this is on, dark matter is always added directly to the BH, while stars add to the accretion disk if SINK_ALPHADISK_ACCRETION is on, otherwise to the BH (but does not contribute towards feedback or the Eddington limit, i.e. acting as if it is not tidally disrupted). Note that SINK_GRAVCAPTURE_NONGAS can be turned on with other models (below) for gas accretion. If you use this module, cite Hopkins et al., 2016, MNRAS, 458, 816 (where this was developed and studied in detail).

SINK_GRAVACCRETION: Enables the “unresolved limit” accretion rate estimator based on gravitational torques from Hopkins & Quataert 2011, MNRAS, 415, 1027. This is a general family of accretion rate estimators based on the assumption that the rate-limiting step for accretion is angular momentum transport (as opposed to the traditional Bondi-Hoyle-type estimators, which do not include any accounting for angular momentum and/or cooling). The fundamental assumptions of the default models here (values =0/1) are tested and designed to reproduce the accretion rates through annuli ranging from accretion disk scales <~0.01 pc through galactic ~1000 kpc scales, with a primary focus on scales ~1–100 pc. The dominant mechanism of angular momentum transfer on larger scales are torques due to inhomogeneities (e.g. due to instabilities, clumps, etc) in a gas+stellar disk. See that paper for details. In general this accretion rate estimator can be more accurate by a factor of \(\sim 10^{4} - 10^{8}\) compared to e.g. the Bondi estimators below (on scales outside the inner accretion disk, which this is not designed to reproduce). Quantities such as the gas angular momentum around the sink are calculated on the fly and used in the accretion rate estimation. Two variants here (setting =0 or =1) use two slightly different estimators to calculate the angular momentum and disk-mass-fraction around the sink (which enter the accretion rate determination), and evaluate at either the adaptive kernel radius or a fixed physical radius, as stated. For [=2] the scaling is reduced to simply assume a constant efficiency per dynamical time. For [=3], this uses the simpler classic gravito-turbulent scaling for the properties of the outer disk, which is reasonable for some circumstellar disks and can apply if the medium is gas-dominated in an external potential, thin, has toomre Q above unity, and can be approximated by a beta-cooling model without cooling too fast. For [=4], the fixed efficiency is again assumed but now set to fixed at the sink radius of influenced defined in the usual fashion from the surrounding material kinematics. For [=5] a hybrid model is employed, where the circularization radius of gas is explicitly calculated based on its angular momentum content, and if this is sufficiently low, where the Bondi-Hoyle limit could actually apply, the Bondi formula is used. [=6] and [=7] employ scalings which interpolate between fixed accretion per dynamical time for bound gas, and Bondi-Hoyle, appropriately for a density profile scaling as \(\rho\propto r^{-1}\) or \(r^{-2}\), respectively, in the self-gravitating component. [=8] Uses the estimator from Hubber et al. 2013 for a toy alpha-disk model which is regulated by bondi-accretion, valid if the bondi limit applies to large radii and the disk is a thermal-pressure-dominated alpha disk in steady state with an approximately constant thermal scale-height-to-radius ratio throughout. Values [=9,10,11] adopt the classic Bondi-Hoyle formula, i.e. neglect angular momentum and assume spherical accretion and no cooling. Note that a large number of studies have shown this is not a good approximation to periods of high sink or stellar accretion, since it assumes the gas has no angular momentum (when, in fact, understanding gas accretion onto sinks from large scales is primarily an angular momentum problem). Contrary to some claims in the literature, there is no Bondi-Hoyle formula that accounts for angular momentum – the actual scalings in the angular-momentum dominated regime resemble the gravito-turbulent and gravitational-torque accretion models discussed above, which have qualitatively different dimensional scalings (nearly independent of sink mass and sound speed, for example, where Bondi-Hoyle depends strongly on both of these). Still, the Bondi-Hoyle limit is potentially relevant for either situations (1) where the sink is accreting smoothly from a hot, hydrostatic, pressure-supported atmosphere, or (2) where a ‘seed’ sink is moving through the ISM (on scales where it does not strongly influence the potential), closer to the regime the Bondi-Hoyle accretion theory was designed to represent. The notes_blackholes file has additional details of the scalings adopted. The [=9] implementation is standard Bondi-Hoyle scaling, accounting for sink-gas relative velocities (the denominator is \((c_{s}^2 + |{\bf v}_{\rm gas}-{\bf v}_{\rm BH}|^2)^{3/2}\)). Set [=10] to ignore the relative velocity and use just the sound speed. Set (=11) to slightly modify the pre-factor in front of the Bondi rate (giving it a power-law density dependence), which will exactly Booth & Schaye 2009, and subsequent papers (OWLS, EAGLE, etc.) – that particular sub-version requires GALSF also be set.

SINK_GRAVACCRETION_STELLARFBCORR: Multiple the accretion rate by the appropriate function to account for additional acceleration-dependent retention from stellar feedback in Mdot. This is relevant in simulations where the accretion rate onto the BH is estimated from scales which are still star-forming (larger than 0.01 pc or so, for super-massive black holes). Stellar feedback will expel some of that material, suppressing the accretion rate relative to a model neglecting stellar feedback, in a way that depends on the acceleration scale. If you are using this flag, cite Hopkins et al., arXiv:2103.10444, for both the analytic derivation of these scalings and the numerical methods/implementation.

Note that you should only activate at most one of the flags SINK_BONDI, SINK_GRAVACCRETION, or SINK_GRAVCAPTURE_GAS. These are coded assuming one determines the model for the BH accretion rate of gas, and will give incorrect or un-predictable results if more than one are enabled. You can, however, enable SINK_GRAVCAPTURE_NONGAS “on top of” any of these gas accretion models.

BH Feedback Models

BHs also act through a variety of “feedback” channels. GIZMO includes a range of detailed hydrodynamic and radiative feedback models for BHs. However, just like for stellar feedback, some of these are under active development: these will be clarified below.

These models also involve their own parameterfile settings, in addition to those required for the BH accretion models. These are detailed below. 

####################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- feedback models/options
## ----------------------------------------------------------------------------------------------------
#SINK_FB_COLLIMATED               # BH feedback is narrowly collimated along the axis defined by the angular momentum accreted thus far in the simulation. Cite Su et al. arXiv:2102.02206
# -- thermal (pure thermal energy injection around BH particle, proportional to BH accretion rate)
#SINK_THERMALFEEDBACK             # constant fraction of luminosity coupled in kernel around BH. cite Springel, Di Matteo, and Hernquist, 2005, MNRAS, 361, 776
# -- mechanical (wind from accretion disk/BH with specified mass/momentum/energy-loading relative to accretion rate)
#SINK_WIND_KICK=1                 # gas in kernel given stochastic 'kicks' at fixed velocity. (>0=isotropic, <0=collimated, absolute value sets momentum-loading in L/c units). cite Angles-Alcazar et al., 2017, MNRAS, 464, 2840
#SINK_WIND_SPAWN=2                # spawn virtual 'wind' particles to carry BH winds out. value=min number spawned per spawn-step. Cite Torrey et al 2020MNRAS.497.5292T, Su et al., arXiv:2102.02206, and Grudic et al. arXiv:2010.11254 for use and numerical methods and tests
# -- radiative: [FIRE] these currently are built on the architecture of the FIRE stellar FB modules, and require some of those be active. their use therefore follows FIRE policies (see details above). however, if explicit-radiation-hydrodynamics is enabled, users can achieve this functionality entirely in the public code, with appropriate hooks in the cooling functions
#SINK_COMPTON_HEATING             # enable Compton heating/cooling from BHs in cooling function (needs SINK_PHOTONMOMENTUM). cite Hopkins et al., 2016, MNRAS, 458, 816
#SINK_HII_HEATING                 # photo-ionization feedback from BH (needs GALSF_FB_FIRE_RT_HIIHEATING). cite Hopkins et al., arXiv:1702.06148
#SINK_PHOTONMOMENTUM              # continuous long-range IR radiation pressure acceleration from BH (needs GALSF_FB_FIRE_RT_LONGRANGE). cite Hopkins et al., arXiv:1702.06148
# -- cosmic ray: these currently build on the architecture of the SINK_WIND modules, one of those must be enabled, along with the usual cosmic-ray physics set of modules for CR transport. the same restrictions apply to this as CR modules. developed by P. Hopkins
#SINK_COSMIC_RAYS                 # explicitly inject and transport CRs from BH. set injection energy efficiency. injected alongside mechanical energy (params file sets ratios of energy in different mechanisms)
####################################################################################################

SINK_FB_COLLIMATED: BH feedback is narrowly collimated along the axis defined by the angular momentum accreted thus far in the simulation. This requires SINK_FOLLOW_ACCRETED_ANGMOM, which it uses to define a preferred axis, and feedback is narrowly collimated along those axis. We say ‘narrowly’ because resolution and physical expansion of the jets/radiation/cosmic rays will quickly dominate the injection along a ray, as it is done in-code. This modifies all AGN feedback mechanisms, including mechanical, jets and radiation.

SINK_THERMALFEEDBACK: Enables the GADGET/AREPO AGN feedback model, where a fixed fraction of the BH luminosity (set in the parameterfile as SinkFeedbackFactor) is coupled to the surrounding gas as purely thermal energy, distributed among the gas particles within the kernel of the BH (the same ones that determine the BH accretion rate) in a kernel-weighted fashion. Users should cite Springel, Di Matteo, and Hernquist, 2005, MNRAS, 361, 776.

SINK_WIND_CONTINUOUS/SINK_WIND_KICK/SINK_WIND_SPAWN: These enable mechanical feedback from BH particles. This corresponds to observed ubiquitous AGN winds with velocities ~500–30,000 km/s, or hydrodynamic jets (if collimated). This is intended to represent the winds not at their initial launching radius, but on larger scales where the winds become hydro-dynamic, and so their basic properties are summarized by two parameters: a mass loading (\(\dot{M}_{\rm wind} / \dot{M}_{\rm BH}\)) and launch velocity. These are set in the parameterfile, and define the time-dependent mass, momentum, and energy flux. The “notes” document for sink particles/black holes explains in detail how these relate to physical values. The parameterfile must include SinkFeedbackFactor, which multiplies the mass and momentum-loading of the wind (so default=1), Sink_accreted_fraction (or Sink_f_accretion which does the same thing), which determines the “base” mass-loading of the wind (ratio of what goes into the BH versus wind at the accretion disk; this is then multiplied by SinkFeedbackFactor to effectively determine the total mass-loading of the wind on coupling scales), and Sink_outflow_velocity (or Sink_v_outflow), which determines the wind launch velocity. The difference between the modules is in how they operate, algorithmically.

SINK_WIND_KICK: Gas in kernel given stochastic “kicks” at fixed velocity. Set <0 for isotropic (radial) kicks, >0 for collimated kicks (all kicks along axis of angular momentum vector of material inside the BH kernel). For historical compatibility, the absolute value of this parameter can be set to an arbitrary value, which sets the momentum-loading of the wind at coupling in units of L/c from the BH – however this is redundant with the SinkFeedbackFactor parameter (do not set both). If you use this module, cite Angles-Alcazar et al., 2017, MNRAS, 464, 2840.

SINK_WIND_SPAWN: This will spawn ‘wind’ particles to carry BH winds out. This is the most flexible and accurate algorithm, allowing arbitrary collimation (e.g. jets), and allowing particles to carry arbitrary combinations of thermal/kinetic/magnetic/cosmic ray energies, and allowing one to accurately capture reverse shocks (which are otherwise implicitly lost in the continuous and kicking strategies, unless the mass inside the free-streaming radius of the outflow is extremely well-resolved). However it is also numerically the most expensive and requires care in use. In addition to parameters above, user must set Cell_Spawn_Temperature (or the alternative variable name Sink_outflow_temperature which does the same thing; this determines the internal temperature of the spawned wind, in K) and Sink_outflow_particlemass (which sets the mass of the virtual particles, in code units). Note that if the Config flag SINK_SCALE_SPAWNINGMASS_WITH_INITIALMASS is set, then the spawned mass of new cells is not fixed to an absolute value, but to a ratio relative to the mass of the particle when the sink particle first formed (useful for simulations with variable resolution): in this case the flag Sink_outflow_particlemass should be replaced with Cell_Spawn_Mass_ratio which sets the mass of spawned cells in units of the initial sink mass. The value of SINK_WIND_SPAWN here is the number of new elements spawned simultaneously. It is recommended this be set to an even number, as the new elements will then be spawned at alternating antipodal points and directions, ensuring that the algorithm conserves center-of-mass position and linear momentum exactly (not simply statistically). The methods were developed in Torrey et al 2020MNRAS.497.5292T, and applied and extensively tested for AGN in Su et al., arXiv:2102.02206 and for protostellar jets in Grudic et al. arXiv:2010.11254. While this is part of the public code, care is needed: arbitrarily spawning cells without special care for how their properties are initialized (especially with additional physics like MHD, cosmic rays, radiation-hydrodynamics) can lead to unphysical values and crashes easily. So users are strongly encouraged to contact one of those authors and discuss the module with them to determine how best to apply it stably to their problems, without causing numerical errors.

SINK_COMPTON_HEATING: Turn on to enable Compton heating/cooling of gas by the BH radiation field, following Sazonov et al. 2004. Since heating/cooling is dominated by hard X-ray/IR photons, respectively, the heating spectrum is simply scaled by an inverse square law, with a fixed Compton temperature ~3e7 K (see that paper for details). We also follow Faucher-Giguere & Quataert 2012 and do not allow the Compton+free-free cooling rate to exceed the Coulomb energy transfer rate between ions and electrons calculated for an ion temperature T in the limit where the electrons are efficiently cooling. Requires COOLING on. This requires SINK_PHOTONMOMENTUM is turned on (so the code can propogate the BH flux), but is independent of the strength you set for SINK_PHOTONMOMENTUM in the parameterfile. Note that this uses some algorithms which are still in development: please contact their developers before developing or using the other connected/dependent modules on their own. If you use this module, cite the methods and physics test paper where it was presented, Hopkins et al., 2016, MNRAS, 458, 816.

SINK_HII_HEATING: Turn on to enable BH photo-ionization feedback. This requires GALSF_FB_FIRE_RT_HIIHEATING (the stellar photo-ionization feedback module) is also enabled. Then photo-ionization is treated from the BH identically to that from the stars, modulo the shape and magnitude of the ionizing spectrum, which is calculated for the BH luminosity following the empirical quasar spectral templates in Hopkins, Richards, & Hernquist 2007. This uses the FIRE HII-heating algorithms and implementation (Hopkins et al., arXiv:1702.06148), so its use follows the FIRE citation guidelines [see notes above for the FIRE modules in the stellar feedback section].

SINK_PHOTONMOMENTUM: Turn on to enable BH radiation pressure feedback. This follows the mechanisms used to couple stellar radiation pressure to the gas, but with additional corrections for the non-isotropic geometry of the BH accretion disk and obscuration from the torus region, including single-scattering at the dust sublimation radius and IR scattering outside this; this follows the fitting functions taken directly from full radiative transfer calculations by N. Roth (private communication, but see Roth et al. 2012, arXiv:1204.0063, for details). You can set the strength of this mechanism in the parameterfile separately from the other BH feedback mechanisms with the multiplier Sink_Rad_MomentumFactor (default =1). This uses the FIRE radiation pressure algorithms and implementation (Hopkins et al., arXiv:1702.06148), so its use follows the FIRE citation guidelines [see notes above for the FIRE modules in the stellar feedback section].

SINK_COSMIC_RAYS: Turn on to enable BH feedback via cosmic rays, with explicit CR transport. Requires COSMIC_RAY_FLUID and appropriate choices there for CR transport. Also requires one of the BH mechanical feedback modules. CRs will be coupled alongside mechanical energy, with specified efficiency (so you can make it ‘pure’ CR by making mechanical energy weak, CRs large). From then on, trated as ‘normal’ CRs.

BH Output/Additional Options

These are miscellaneous additional BH options. See also the description of the various modules SINGLE_STAR_TIMESTEPPING, SINK_ACCRETE_NEARESTFIRST, SINK_RETURN_ANGMOM_TO_GAS, SINK_SWALLOW_SMALLTIMESTEPS, above in the ‘Sink Particle and Single-Star’ modules: these are all fully-compatible with the BH modules here as well!

####################################################################################################
## ----------------------------------------------------------------------------------------------------
# ----- output options
## ----------------------------------------------------------------------------------------------------
#SINK_OUTPUT_MOREINFO             # output additional info to "sink_details" on timestep-level, following Angles-Alcazar et al. 2017, MNRAS 472, 109 (use caution: files can get very large if many BHs exist)
#SINK_CALC_DISTANCES              # calculate distances for all particles to closest BH for, e.g., refinement, external potentials, etc. cite Garrison-Kimmel et al., MNRAS, 2017, 471, 1709
## ----------------------------------------------------------------------------------------------------
####################################################################################################

SINK_OUTPUT_MOREINFO: Outputs a number of additional details and useful information to the sink_details files tracking individual BH particles. Coded by Daniel Angles-Alcazar. These output BH information at the simulation-timestep level, so BH histories can be re-constructed – this is particularly useful for modeling BH evolution with e.g. different accretion models in post-processing, following the methodology in Angles-Alcazar et al. 2017, MNRAS 472, 109. One important caveat: if your simulation has a very large number of BH particles, these files can quickly blow up to enormously large values, so be careful.

SINK_CALC_DISTANCES: Calculates the position of the nearest BH particle at all times, to all particles, and their position relative to it. This can be useful for various feedback algorithms, but also for on-the-fly refinement criteria (for example, super-Lagrangian refinement schemes which depend on the distance to the BH), or for inputing analytic gravitational potential terms (e.g. if one wishes to evolve a background halo+disk potential which is smooth, in a simulation focused on detailed gas only around the BH, one can treat them analytically using this approach. If you use this for science results, please cite Garrison-Kimmel et al., MNRAS, 2017, 471, 1709 where it was developed (and used for the gravitational-potential models).

Radiative Cooling & Chemistry

#############################################################
# ---- Radiative Cooling & Thermo-Chemistry
#############################################################

These modules enable different treatments of explicit radiative cooling and thermo-chemistry in GIZMO simulations. They can allow for optically thin and optically thick cooling limits, complicated ion+atomic+molecular gas chemistry in both equilibrium and non-equilibrium regimes, explicit hooks into the various radiation-hydrodynamic modules above (to alter radiative heating rates or track the cooling radiation explicitly), and a choice of different external libraries (if desired) to solve the cooling. The modules here are generally designed to be applicable for gas-phase chemistry relevant at densities \(\ll 10^{-6}\,{\rm g\,cm^{-3}}\), or \(n_{H}\ll 10^{18}\,{\rm atoms\,cm^{-3}}\), i.e. densities from the lowest-density IGM to high-density regions of proto-planetary and circum-planetary disks. But they are not designed for regimes such as planetary or stellar interiors (where other heating processes like nuclear burning or radioactive decay may dominate) – in those cases, other modules in GIZMO are designed to give more appropriate behavior. Proper citations for the different sub-modules are given below, but all users should cite Hopkins et al. 2017 (arXiv:1702.06148), where Appendix B details the default cooling physics as of 2018 (though various updates have since been pushed, as detailed below, and many more extended/detailed options exist).

We stress that – provided the correct flags have been enabled – GIZMO contains several different detailed thermo-chemical networks with similar complexity to other ‘standard’ works in the field (e.g. Glover and Clark, Wise and Abel, and many others) and these networks have all been cross-validated in extensive detail. But of course, if one does not enable the relevant physics for your problem, the thermo-chemical network will not necessarily give reasonable results.

Built-In Cooling & Thermochemistry Physics

##------------ Top-level switches -------
#COOLING                        # top-level switch to enable radiative cooling
#METALS                         # top-level switch to enable tracking metals
#DISK_BETA_COOL                 # simple beta-cooling for disks (exclusive with COOLING)
##---------------------------------------

COOLING: This is a top-level switch which enables radiative cooling and heating and ionized+atomic+molecular gas chemistry. If no other cooling libraries (e.g. GRACKLE or CHIMES, or the more extended networks within the ‘default’ code as described below) are enabled, this also supplies the default cooling and chemistry libraries written by Phil Hopkins. The standard cooling module is designed for applications like star and galaxy formation simulations but has been extended to planet formation and other contexts, and so assumes by default optically thin radiative cooling, with approximate corrections for the optically thick regime assuming a local slab geometry with a Sobolev-like estimator. The default module includes a huge range of processes, including heating+cooling from H and He ionization+recombination, collisional, free-free, molecular, dust collisional, cosmic ray, and Compton effects. With this enabled, the ionization+recombination requires an assumption for the UV background, specified in tabular format in the “TREECOOL” file (which must be placed in the run directory, i.e. the run-time directory where the compiled binary resides). A copy of this file is included in the source code itself (in the cooling directory) and on the GitHub wiki page. The computed cooling rates effectively extend from \(\sim 10^{4}\) K to \(10^{9}\) K. Other heating and cooling processes including local radiative heating, photo-electric heating, and fine structure heating/cooling, metal line cooling, nuclear (fission/fusion) heating, and more can be enabled by other compile-time switches (they are not automatically included in this switch). Any cooling associated with metal species requires METALS be enabled and particles have a defined metallicity field. Anyone using this should cite Hopkins et al. 2017 (arXiv:1702.06148); a complete list (and description of the physics and fitting functions) of all cooling physics included is given in their Appendix B [the parts enabled by this piece of the code include all components/terms above \(10^{4}\) K, except for the COOL_METAL_LINES_BY_SPECIES terms described below].

The TREECOOL file uses the compiled UV background data from Claude-Andre Faucher-Giguere’s paper: the current default follows Faucher-Giguere, 2020, MNRAS, 493, 2, 1614, “A cosmic UV/X-ray background model update”, appropriately modified following Onorbe et al., 2017, ApJ, 837, 2, 106, to reionize at the ‘correct’ target redshift (this owes to a difference in how the analytic models like Faucher-Giguere et al. treat the neutral fraction in the model, versus how in the simulations with a TREECOOL file the reionization background is applied everywhere uniformly, which can inadvertently lead to earlier reionization than intended). This also includes the updated contribution from AGN following Shen et al., 2020, arXiv:2001.02696, ‘The Bolometric Quasar Luminosity Function at z=0–7’. If you use this file, delete the relevant “header” lines in TREECOOL (preceded by ##), make sure the file is simple named TREECOOL and included in your run-time directory with the compiled GIZMO binary, and make sure to cite the papers above. Before June 2020, the default TREECOOL file included with GIZMO was based on an earlier version of the UV background, from Faucher-Giguere et al., 2009, ApJ, 703, 1416.

METALS: Enables tracking metallicities/passive scalars associated with gas and stars/sinks. Enrichment will be handled by the different specific physics modules below (e.g. some of the star formation modules attempt to inject metals on-the-fly), or can be added to the initial conditions (or specified in the parameterfile, if you wish to initialize the simulation to a uniform metallicity and set of abundance ratios). This specific flag does not require GALSF be enabled; if you turn it on without GALSF metals are treated as fixed by the ICs or passive scalars which can diffuse if e.g. models for turbulent metal diffusion are enabled, and can act in the metal-line cooling routines (they just do not have a dynamical mechanism for enrichment). Citation is not required for the specific passive-scalar routines, but the relevant cooling and diffusion routines do ask for appropriate citations.

DISK_BETA_COOL: Enables a minimal ‘beta-cooling’ prescription for idealized protoplanetary-disk problems. It is an alternative to, and mutually exclusive with, COOLING (the code will not compile with both). Once per timestep each active gas cell’s specific internal energy is relaxed toward a constant irradiation floor, \(u_{\rm irr}\), as \(du/dt = -(u-u_{\rm irr})/t_{\rm cool}\) with \(t_{\rm cool} = \beta/\Omega\). The update is integrated exactly as \(u_{\rm new} = u_{\rm irr} + (u_{\rm old}-u_{\rm irr})\,\exp(-\Delta t/t_{\rm cool})\). With BOX_SHEARING, \(\Omega\) is the shearing-box value at the box center. Otherwise the code assumes a central potential at the origin and sets \(\Omega = (|{\bf a}_{\rm grav}|/r)^{1/2}\) from each cell’s gravitational acceleration and distance from the origin. Requires the parameters BetaCool_Beta (the dimensionless \(\beta\)) and BetaCool_Tirr (the floor temperature in K, converted to \(u_{\rm irr}\) with the default mean molecular weight; set =0 for pure cooling toward zero, subject to MinGasTemp). This is deliberately a crude approximation. For realistic disk thermodynamics, use explicit radiation-hydrodynamics with dust-gas thermal coupling (e.g. RT_OPACITY_FROM_EXPLICIT_GRAINS) instead.

##------------ Additional options within default (Hopkins+17) COOLING
#COOL_METAL_LINES_BY_SPECIES    # use full multi-species-dependent
#COOL_LOW_TEMPERATURES          # fine-structure+molecular+optically thick cooling
#COOL_MOLECFRAC=1               # choose how to track molecular H2 fraction (1-6, see docs for details) 
##---------------------------------------

Within the default COOLING module there are many sub-modules. Some of these depend on enabling specific radiation-hydrodynamics modules, discussed below. For example, if photo-electric or Lyman-Werner or Compton (X-ray) or dust infrared or ionizing wavelengths are evolved explicitly in the radiation-hydrodynamics, they all automatically are incorporated into this COOLING routine. However there are some additional switches here that are important for many simulations. All of the options below are part of the public code, but please follow the relevant citation guidelines.

COOL_METAL_LINES_BY_SPECIES: Uses full multi-species-dependent metal-line cooling tables. The tables correspond to CLOUDY runs following Wiersma et al. 2009 (MNRAS, 393, 99). The relevant pre-computed tables can be downloaded here or on the GitHub wiki page. This requires that METALS and COOLING are on. When active, the code will follow 11 species independently (H, He, C, N, O, Ne, Mg, Si, S, Ca, Fe, in addition to the “total” metallicity), with heating/cooling computed for each assuming its optically thin (and self-consistently computing the ionization states). Users should cite Wiersma et al. 2009 (MNRAS, 393, 99) and Hopkins et al. 2017 (arXiv:1702.06148). To use this module, one must download and unpack the actual cooling tables and have them (un-zipped and un-tarred) in a folder named spcool_tables placed within the run-time directory of GIZMO. Again, tables can be download either here or on the GIZMO public or private (either one) GitHub code repository website.

COOL_LOW_TEMPERATURES: Adds fine-structure and molecular cooling, to extend the cooling curves down to ~10 K. The cooling rates at low-temperatures are explicitly metal-species-dependent if COOL_METAL_LINES_BY_SPECIES is also enabled (any version of this requires METALS enabled). These follow a detailed ensemble of processes including molecular and fine structure cooling, dust-gas collisional cooling, cosmic ray heating, photo-electric and other terms. Some terms are taken from fits to CLOUDY models, but most are separated because each can depend on the local abundances and radiation fields in different ways. The list of terms, and the sources for each expression used, are given in arXiv:1702.06148 and updated in arXiv:2203.00040. This requires COOLING is enabled; any cooling associated with metal species requires METALS be enabled and cells have a defined metallicity field. Note that while this includes an approximate treatment of optically-thick cooling, it is designed for simulations of star and galaxy formation and should be extrapolated to dense systems (e.g. stars, planetary atmospheres, etc) with caution and does not even attempt to include physics such as neutrino cooling. Anyone using this should cite Hopkins et al. 2017 (arXiv:1702.06148) and 2023 (arXiv:2203.00040); a complete list (and description of the physics and fitting functions) of all cooling physics included is given in the Appendix B of the former paper [the parts enabled by this piece of the code include all components/terms below \(10^{4}\) K] and since 2020 use the updates given in section 3 of the latter paper.

COOL_MOLECFRAC: If COOL_LOW_TEMPERATURES is enabled, the code will extend the ionized+atomic molecular network to include Hydrogen (H2) molecular chemistry. This switch (set to a given value) determines how the H2 fractions and cooling rates are estimated. Generally speaking, higher values involve more complicated models for H2, which may require more additional fields are set as they also depend on quantities like the local abundances, dust-to-gas ratio, FUV radiation field, etc. The values here correspond to: (1) Default to an extremely-oversimplified scaling for H2 fraction as a function of density and temperature, set by fits to the temperature below which gas at a given density becomes molecular from cloud simulations presented in Glover+Clark 2012. (2) Estimate H2 fraction with Krumholz & Gnedin 2010 fitting function, assuming simple scalings of radiation field, clumping, and other factors with basic gas properties so this is a function only of surface density (estimated with a Sobolev-type approximation) and metallicity, truncated at low values (or else it gives non-sensical answers). (3) Estimate H2 fractions from the approximate large-scale expressions from Gnedin & Draine (2014ApJ…795…37G). use the sub-grid final expression calibrated to ~60pc resolution simulations with equilibrium molecular chemistry and post-processing radiative transfer from Gnedin & Draine 2014 (See their Eqs. 5–7). This depends on density, metallicity, column density, and the local FUV field. (4) Estimate H2 fractions from the approximate large-scale expressions from Krumholz, McKee, & Tumlinson (2009ApJ…693..216K). This is an equilibrium model of spherical clouds of internally uniform properties exposed to incident radiation. Depends on density, column density, metallicity, and incident FUV field. (5) Estimate molecular fractions for thermochemistry+cooling from local equilibrium H2 formation+destruction with clumping and self-shielding (Hopkins et al. arXiv:2203.00040). This solves the full set of H2 formation and destruction equations from Nickerson, Teyssier, & Rosdahl et al. 2018 (including dust and gas-phase formation, 3-body formation and collisional H-H2 and H2-H2 dissociation, Lyman-Werner and ionizing and cosmic ray dissociation). For the radiation terms this accounts for line self-shielding with turbulent and cell line-blanketing terms from Gnedin & Draine 2014, and dust self-shielding in the radiation hydro module, with clumping factors and the turbulence model using a combination of the code local turbulent (eddy) estimator plus a standard lognormal distribution at the cell scale. The network is solved assuming local equilibrium fully-implicitly. (6) This network is identical to (5), the only difference is that the H2 fraction is explicitly evolved, allowing it to have non-equilibrium values. Citations for these stem from Hopkins et al. 2023, 2023MNRAS.519.3154H or arXiv:2203.00040 – cite that paper if you use any of these.

COOL_UVB_SELFSHIELD_RAHMATI: This replaces the self-shielding approximation used for the UV background with a slightly different but more accurate approximation (Hopkins et al. arXiv:2203.00040). This follows Rahmati et al. 2013MNRAS.431.2261R, who calibrate to variation radiation-transport calculations, but with an additional simple correction to account for the fact that the fitting functions from that paper decline (unphysically) too slowly at high gas densities, which leads to artificial suppression of molecular gas even at arbitrarily high densities.

Note that there has unfortunately been some confusion about the physics and applications of the GIZMO low-temperature cooling modules. Specifically, Wibking and Krumholz 2023 (WK23) stated (incorrectly) that there was an ‘error’ in the implementation of cooling in GIZMO, and showed some cooling curves in their Appendix A which were reproduced directly in e.g. Kim et al. 2023 (arXiv:2210.08024). But the issue noted in WK23 was not in fact, as the authors acknowledge in the text, an error or bug in the code – indeed, the authors validated directly that GIZMO was, in their test problem, solving the thermochemistry correctly given the input physics/assumptions for the flags turned on. Specifically, WK23 considered a test problem where they simulated a homogeneous box illuminated by a constant background field (no local sources, no explicit or implicit radiation transport), turning on the pre–2020 version of the COOL_LOW_TEMPERATURES flag with no other physics (not including any soft X-ray, cosmic ray, or shock/hydrodynamic sources of ionization, nor ionization of heavier species beyond H and He), using the popular self-shielding prescription generally used for cosmological simulations (across many different codes) at the time derived from e.g. Schaye et al. 2001 (2001ApJ…559..507S) to shield that uniform background. WK23 (Appendix A) showed that this leads – as it should by construction – to exponential suppression of the free electron fraction at high densities in primarily neutral gas. They then showed that this (via its indirect effects on e.g. photo-electric heating) has relatively modest effects on the equilibrium temperature and pressure of the neutral, dense ISM at densities \(n_{\rm H} >1\,{\rm cm}^{-3}\) and temperatures \(T<10^{4}\,{\rm K}\), equivalent to shifting \(\langle T(n_{H})\rangle\) by a factor of a few in \(n_{H}\) or \(T\), compared to a model which included no self-shielding whatsoever (only optically-thin treatment of the meta-galactic and galactic background in the cold, dense gas). This is again expected, and users should note this is generally smaller than other physical effects which GIZMO includes options for, such as different choices of how to approximate the molecular gas formation/destruction chemistry, treatment of the radiation fields and their thermo-chemical coupling, explicit cosmic ray physics, etc. Unfortunately this cooling curve was labeled ‘FIRE–2 Cooling’ in WK23, despite the fact that the physics and numerical module choices used in their setup are quite different from what is actually enabled in the published FIRE–2 papers and, as noted in e.g. arXiv:1702.06148, 1811.12462, 2203.00040 and other papers, the thermal pressure/temperatures in the WNM and CNM are actually much closer in actual FIRE–2 simulations to what WK23 argued was their best ‘full physics’ reference model in their paper (interested users can obtain the public FIRE–2 snapshots available through http://fire.northwestern.edu). Moreover, post–2020, the default behavior of the COOL_LOW_TEMPERATURES flag in GIZMO has been extended with a variety of physics detailed and extensively validated against state-of-the-art chemical networks including those in Glover and Clark, Wise and Abel, GRACKLE, CHIMES, KROME, and many more, explicitly for neutral gas at densities as high as \(10^{18}\,{\rm cm}^{-3}\) in 2021MNRAS.506.2199G, 2023MNRAS.519.3154H. This includes explicit treatment of sources of ionization like cosmic rays, extending the radiation networks enabled by any of the user-desired ionization-related modules (and of the relevant RHD modules including those used in FIRE/STARFORGE and many other projects) to include ionizations from the entire frequency range, expanding the treatment of hydrodynamic/shock ionization (which was included pre–2020 but not in the specific test problem in WK23) to non-equilibrium shock ionization, self-consistent ionization of heavier elements, and inclusion many other relevant physics in these regimes such as electron depletion onto dust grains, detailed cross-coupling between molecular and dust chemistry, H- anion chemistry, and more. Of course, users also have the option to turn off or modify the self-shielding treatment entirely, or to use the more modern self-shielding calibration from Hopkins et al. MNRAS 2023 519 2 3154, or (ideally if important to the problem) to simply use self-consistent radiation transport with local sources, since the fundamental argument of WK23 was not specific to GIZMO (nor FIRE) but rather to the popular Schaye et al. 2001 self-shielding prescription.

However, we generally note that many dozens of papers (including, but certainly not limited to, GIZMO papers such as 2012MNRAS.421.3488H, 2014MNRAS.445..581H, 2018MNRAS.478.3653O, 2018MNRAS.480..800H, 2020MNRAS.491.3702H, 2023MNRAS.519.3154H) have shown that in most cosmological and/or galaxy formation simulations, orders-of-magnitude larger changes than those noted in WK23 (including simply turning off cooling entirely below \(10^4\) K) have extremely minor effects on almost all galaxy dynamical and star formation properties (and 2023MNRAS.519.3154H specifically compared results including/excluding the terms discussed in WK23). For individual star formation simulations using e.g. the SINGLE_STAR_DYNAMICS modules, or if the primary focus of your study is predicting these atomic/molecular properties in detail, then of course more detailed/accurate treatments are desired and recommended. For these, we recommend using the modern ‘full-physics’ version of GIZMO, but (crucially) avoiding the ambiguity of simple self-shielding prescriptions entirely by using one of the many available explicit radiation-hydrodynamics treatments in the code (which are coupled directly to these thermochemistry modules).

Grackle Thermochemistry Libraries

Grackle is an alternative chemical network using external libraries for solving thermochemistry+cooling. These treat molecular hydrogen, in particular, in more detail than our default networks, and so can be more accurate for ‘primordial’ (e.g. 1st-star) gas. But they also have less-accurate treatment of effects such as dust-gas coupling and radiative feedback (Compton and photo-electric and local ionization heating) and high-optical-depth effects, so are usually less accurate for low-redshift, metal-rich star formation or planet formation simulations, depending on which of the modules above you choose to enable (or not).

##---------------------------------------
## ---- GRACKLE: alternative chemical network 
#COOL_GRACKLE                   # enable Grackle: cooling+chemistry package
#COOL_GRACKLE_CHEMISTRY=1       # choose Grackle cooling chemistry
#COOL_GRACKLE_APIVERSION=1      # set the version of the grackle api
##---------------------------------------

COOL_GRACKLE: Enables Grackle. This replaces the default cooling routines with the public cooling and chemistry package available at the url given. Note that this requires the separate installation of the Grackle libraries - these are independent, open source code which is not in any way affiliated with GIZMO. You can get them at: https://grackle.readthedocs.org/en/latest. Once the libraries are installed, and all the relevant paths are appropriately linked (see the lines activated in the Makefile when COOL_GRACKLE is active), this will turn on the relevant code to compile with those libraries and build a set of wrappers to those routines, using them instead of the otherwise default cooling routines. In addition to the standard GIZMO methods paper[s], users of these modules should cite the Grackle methods papers.

COOL_GRACKLE_CHEMISTRY: Chooses the cooling chemistry to be used for the Grackle package. 0=tabular (just use pre-computed cooling tables as a function of temperature and density; the simplest option). 1=Assume atomic chemistry (i.e. actually compute ionization, but not molecular chemistry). 2=atomic plus limited molecular chemistry. 3=atomic plus more extended molecular chemistry (still limited to primordial species). Any cooling associated with metal species requires METALS be enabled and particles have a defined metallicity field. In addition to the standard GIZMO methods paper[s], users of these modules should cite the Grackle methods papers.

COOL_GRACKLE_APIVERSION: This sets the version of the Grackle API to assume is installed and linked at compile-time. The default value here (=1) will work with versions of Grackle before 2.2. After this version, some substantial changes were made to the Grackle API including names and definitions and numbers of variables for core functions, types of pointers/structures which can be passed, etc. So if you are using more recent Grackle versions, you need to set this to a larger integer (=2). Note the newest API is only partially-tested, so you may need to change one or two settings for the modules to compile correctly.

CHIMES Thermochemistry Libraries

CHIMES is another alternative non-equilibrium chemical (ion+atomic+molecular) network, developed by Alex Richings. The core methods are laid out in 2014MNRAS.440.3349R, 2014MNRAS.442.2780R, but have been substantially extended in a number of ways in GIZMO to interact with additional fields that can be present. Unlike Grackle you do not have to install additional libraries here. However, you do have to download additional cooling tables for the code to use.

Per permissions from the lead-developer of CHIMES, Alex Richings, CHIMES is now available as part of the public GIZMO code. Please note that some of the CHIMES sub-modules (optional flags below) couple to different parts of the code written by other authors, and a couple of these may still be in development; these are designated below. Be sure to cite the papers above with the CHIMES methods in any publications that make use of these modules. For additional questions about CHIMES, and additional guidance about whether the modules can be ported into other codes (per their license), please contact Alex Richings to obtain the relevant permissions and or ask questions, as they are the developer.

This has additional hooks to use the various gizmo radiation fields if desired. The modules solve a large molecular and ion network, so can trace predictive chemistry for species in dense ISM gas in much greater detail than the other modules above (at additional CPU cost).

The CHIMES modules in this code have many additional options, only some of the most general are given below. So a special README and Mini-User-Guide documentation is included. In the chimes folder of this GIZMO install (within the cooling directory), please read both README.txt and chimes_gizmo_doc.md before using CHIMES.

SUNDIALS version requirement. CHIMES links the SUNDIALS/CVODE ODE solver and is written against its pre–6.0 API: it calls the context-free forms of CVodeCreate, N_VNew_Serial, SUNDenseMatrix, and SUNLinSol_Dense, and uses the realtype floating type. It therefore requires a SUNDIALS install in the 2.7 - 5.x range. SUNDIALS 6.x made a SUNContext argument mandatory on all of those calls, and 7.x removed realtype (renamed sunrealtype), so building CHIMES against SUNDIALS 6.0 or newer (including current Homebrew/package-manager defaults) will fail to compile without source-level API updates to the CHIMES sources. Point the CHIMESINCL/CHIMESLIBS variables in the Makefile at a SUNDIALS install within the supported range.

##---------------------------------------
## ---- CHIMES: alternative chemical network 
##-----------------------------------------------------------------------------------------------------
#CHIMES                         # top-level switch to enable CHIMES.
#CHIMES_SOBOLEV_SHIELDING       # enables local self-shielding 
#CHIMES_HII_REGIONS             # disables shielding withing HII region
#CHIMES_STELLAR_FLUXES          # couple UV fluxes from the luminosity tree
#CHIMES_TURB_DIFF_IONS          # turbulent diffusions of CHIMES abundances
#CHIMES_METAL_DEPLETION         # uses density-dependent metal depletion
##---------------------------------------
## ------------ CHIMES de-bugging and special behaviors 
##---------------------------------------
#CHIMES_HYDROGEN_ONLY           # hydrogen-only. Ignored if METALS are also set.
#CHIMES_REDUCED_OUTPUT          # full abundances only output in some snapshots
#CHIMES_NH_OUTPUT               # write column densities to snapshots
#CHIMES_INITIALISE_IN_EQM       # initialise CHIMES abundances in equilibrium
##---------------------------------------

CHIMES: Top-level switch required to enable CHIMES and any of the other flags below. Requires COOLING be set above. Also requires COOL_METAL_LINES_BY_SPECIES be turned on if you wish to include metals or dust processes (otherwise this reverts to a primordial network only).

CHIMES_SOBOLEV_SHIELDING: This allows for self-shielding terms for the different species, using the build-in simple Sobolev-type estimator for a shielding length in GIZMO. This sets \(L_{\rm shield} = f_{SL}\,(\Delta x + \rho / |\nabla \rho|)\) where \(\Delta x\) is the local inter-cell separation, and \(f_{SL}\) is the Shielding_length_factor described in the CHIMES documentation.

CHIMES_HII_REGIONS: Disables shielding withing HII region (requires FIRE modules for radiation transport/coupling: uses GALSF_FB_FIRE_RT_HIIHEATING, and citations follow those modules; can be ported to other HII region approximators, but not necessary if explicit ionizing RHD is used).

CHIMES_STELLAR_FLUXES: Tracks stellar fluxes in different age bins through the luminosity tree, using the LEBRON method from FIRE, and couples these to the CHIMES chemistry routines. Note that this should be used instead of FIRE’s standard UV heating routine, if those options are desired with CHIMES.

CHIMES_TURB_DIFF_IONS: Switches on the turbulent diffusion of individual ions and molecules. This is treated in the same way as FIRE’s standard metal diffusion option. Requires TURB_DIFF_METALS and TURB_DIFF_METALS_LOWORDER. Citations follow the citations policy for those modules.

CHIMES_METAL_DEPLETION: Reduces the abundance of metals in the gas-phase according to observed density-dependent metal depletion factors from Jenkins (2009) and De Cia et al. (2016). It also computes a density-dependent dust to gas ratio that is consistent with these depletion factors.

CHIMES_HYDROGEN_ONLY: Intended to run the CHIMES network with hydrogen only (hydrogen mass fraction set to 1); ignored if METALS is also set. Note that in the current code no source file reads this flag, so enabling it has no effect on the build.

CHIMES_REDUCED_OUTPUT: By default the full CHIMES abundance array (all species) is written to every snapshot (HDF5 “ChimesAbundances”). With this enabled, the full array is written only to snapshots whose number is a multiple of the parameter N_chimes_full_output_freq. All other snapshots get only a reduced array (HDF5 “ChimesReducedAbundances”) of the electron, HI, H2 and CO abundances. Useful for saving storage in large runs.

CHIMES_NH_OUTPUT: Writes the hydrogen column density of each gas cell to snapshots (HDF5 “ChimesColumnDensity”, in cm${–2}$), using the same Sobolev-type estimate (times Shielding_length_factor) that the chemistry solver uses for self-shielding. This is useful for computing equilibrium abundances in post-processing with exactly the column densities the non-equilibrium calculation used. If OUTPUT_DENS_AROUND_NONGAS is also set, the equivalent effective surface density around star particles is written as well (HDF5 “SigmaEff”, in g cm${–2}$).

CHIMES_INITIALISE_IN_EQM: By default the initial ion and molecule abundances are read from the ICs (or snapshot). With this enabled they are not read. Instead, at startup (any restart flag except 1), each gas cell’s chemistry is evolved at fixed temperature for ten intervals of \(\sim(1/n_{\rm H})\) Myr each (capped at 1 Gyr per interval), bringing the abundances close to chemical equilibrium. Stellar fluxes from the gravity tree are not yet available at that point, so the equilibrium includes only the UV background.

Radiation Hydrodynamics

This set of options controls the explicit radiation-hydrodynamics options in the code. These flags allow you to integrate radiation explicitly or implicitly in the simulations. The code is written to be fully modular: this means you can choose things like the transport algorithm, solver, effective speed of light, frequencies/bands to be tracked, physics of their source functions and opacities and interactions with matter, and chemical networks independently. Any new additions to these modules should follow the same policies. Thus the user can ‘pick and choose’ the set of solvers and wavebands that is most appropriate for their problem. This is recently-developed (and still in progress in some respects) by PFH with extensive input from grad students David Khatami, Mike Grudic, and Nathan Butcher (very special thanks also go to Alessandro Lupi and Clement Bonnerot). Please consult the authors before using any part of the code (some of it may not be fully-debugged, some of it may be currently in-use by a student for an existing project) – some sub-modules are still in development (at least until some of the key methods papers are completed). Additional details, including helpful descriptions of how you, the user, can easily choose your radiation-hydro methods, which equations are actually being solved,

Note that the current implementations include all terms to leading orders in \(O(v/c)^2\), so have Lorentz corrections but do assume the fluid velocities are not highly relativistic (which should be ok since we are not doing relativistic hydro in general). They do include all the relevant radiation-matter coupling terms, corrections to this order from emission/absorption/radiation pressure/back-reaction, and allow for arbitrary absorption and scattering coefficients. More details are found in the notes_rhd file (here).

Radiation Transport Methods

############################################################################################################################
# -------------------------------------- Radiative Transfer & Radiation Hydrodynamics:
# -------------------------------------------- modules developed by PFH with David Khatami, Mike Grudic, and Nathan Butcher (special  thanks to Alessandro Lupi)
# --------------------------------------------  these are now public, but if used, cite the appropriate paper[s] for their methods/implementation in GIZMO
############################################################################################################################
# -------------------- methods for calculating photon propagation (one, and only one, of these MUST be on for RT). whatever method is used, you must cite the appropriate methods paper.
#RT_FLUXLIMITEDDIFFUSION                # RT solved using the moments-based 0th-order flux-limited diffusion approximation (constant, always-isotropic Eddington tensor; fastest, but cannot capture shadows or beaming). cite Hopkins & Grudic, 2018, arXiv:1803.07573
#RT_M1                                  # RT solved using the moments-based 1st-order M1 approximation (solve fluxes and tensors with M1 closure; captures shadows and beaming; more expensive than FLD; currently only compatible with explicit diffusion solver). cite Hopkins & Grudic, 2018, arXiv:1803.07573
#RT_OTVET                               # RT solved using the moments-based 0th-order OTVET approximation (optically thin Eddington tensor, interpolated to thick limit; intermediate between FLD and M1; good for problems dominated by a few point sources). cite Hopkins & Grudic, 2018, arXiv:1803.07573
#RT_LOCALRAYGRID=1                      # RT solved using the exact method of Jiang et al. (each cell carries a mesh in phase space of the intensity directions, rays directly solved over the 6+1D direction-space-frequency-time mesh [value=number of polar angles per octant: N_rays=4*value*(value+1)]. this is still in development, DO NOT USE without contacting PFH
#RT_LEBRON                              # RT solved using the ray-based LEBRON approximation (locally-extincted background radiation in optically-thin networks; default in the FIRE simulations). cite Hopkins et al. 2012, MNRAS, 421, 3488 and Hopkins et al. 2018, MNRAS, 480, 800 [former developed methods and presented tests, latter details all algorithmic aspects explicitly]
############################################################################################################################-

This block controls the algorithm used to actually transport the radiation (indendent of the sources, absorption, bands, effects of the radiation, and even the solver for the transport equations). Because the exact radiation-hydro equations are to expensive to solve on-the-fly in full generality, each module here makes some approximation to simplify the transport equations. Consider carefully whether those approximations are valid or acceptable for your problem – none of these (nor any on-the-fly method in the literature) is truly exact, even in the infinite-resolution limit.

The code here includes several different radiation-hydrodynamics solvers, each of which has been presented or used in various different papers. The core modules are therefore now public. However their development is active and I encourage users who can contribute to the codebase to do so (after checking with the developers of this code), in order to help keep these modules updated and state-of-the-art. Use of any module, of course, requires appropriate citation, even if in the public code. Note that one of the modules must always be chosen to use radiation-hydro.

All of these methods follow Hopkins and Grudic, arXiv:1803.07573 and attempt to follow the face-integrated formulation of the radiative transfer equations for coupling radiation pressure to gas, in order to avoid potentially serious under-estimation of the pressure terms in the vicinity of point sources.

RT_FLUXLIMITEDDIFFUSION: Propagate radiation using the FLD (flux-limited diffusion) approximation. This is a zeroth-order moments closure of the RT equations, reducing them to diffusion, valid in the optically thick limit. Like all moments methods, it cannot reproduce photons moving through one another correctly, but is manifestly photon-conserving. It cannot capture shadowing. Please cite Hopkins and Grudic arXiv:1803.07573.

RT_M1: Propagate radiation using the M1 (first-moment) approximation (Levermore 1984). This expands to one moment higher than FLD/OTVET so a pair of advection equations for flux and energy are solved. It is manifestly photon-conserving and captures shadowing, at the cost of noise and grid effects from propagating vector transport. But like all moments methods, it cannot capture collisionless rays passing through one another. Please cite Hopkins and Grudic arXiv:1803.07573 (for this or any of the other moments-based RHD implementations; but for M1 please also cite Rosdahl et al., 2013, MNRAS, 436, 2188, whose implementation of M1 in RAMSES is an approximate template for the GIZMO implementation).

RT_OTVET: Propagate radiation using the OTVET (optically-thin eddington tensor) approximation (Gnedin+Abel 2001). This is nearly-identical to flux-limited diffusion, in that it is a zeroth-order moments closure of the RT equations valid in the optically thick limit, but it uses the eddington tensor calculated as in the LEBRON method from optically thin media to more accurately capture photon propagation directions in that limit (less accurate in the true optically-thick limit). Like all moments methods, it cannot reproduce photons moving through one another correctly, but is manifestly photon-conserving. It cannot capture shadowing. Please cite Hopkins and Grudic arXiv:1803.07573 (please also cite Gnedin+Abel 2001, New Astronomy, 6, 437, who proposed the OTVET scheme).

RT_LOCALRAYGRID: Propagate radiation using an explicit grid of rays, evolved on top of the spatial mesh of gas resolution elements, following the method of Jiang et al. 2014, ApJS, 213, 7. Essentially, this method integrates the real radiative transfer equations directly in the fully 8-dimensional phase space. Each spatial domain (normal gas resolution element) is a phase-space grid, because in addition to the frequency bins, it carries explicitly a grid of angular directions/rays, each of which has a value of the intensity. The intensity is transported across these rays according to a standard finite-volume method. Unlike the other methods here, this does not make any fundamental assumptions throwing out terms – this is the only method in GIZMO which will, in principle, converge to exact solutions of the full time-dependent radiation-hydrodynamics equations under any circumstances. Of course, this formally requires an infinite number of rays in addition to infinite spatial resolution. The value \(N\) set here is the number of independent polar angles of the grid per octant on the unit sphere – the total number of rays evolved will be \(=4 N (N+1)\) (so setting RT_LOCALRAYGRID=4 means 80 rays are evolved for every frequency bin). So be aware that the computational cost scales steeply with increasing values of this number.

RT_LEBRON: Propagate radiation using the LEBRON (locally-extincted background radiation in optically-thin networks) approximation from Hopkins et al. 2012 (MNRAS 2012, 421, 3488) and Hopkins et al., 2017, arXiv:1702.06148 (where Appendix E gives a complete description of the numerical implementation). This is the default radiation transport approximation in the FIRE simulations, but the pure radiation hydro model is separable from the rest of the FIRE physics (although at the moment, the way it is coded anchors it to the particular set of bands used in the FIRE simulations – it is in principle fully modular to work with the arbitrary set of frequencies here, but this hasn’t been fully-updated yet). See the methods paper from 2017 for a discussion of the actual approximations made in the module. This approximation is distinct from moments-based methods in that it actually correctly allows rays to propagate through one another and exactly reduces to the correct solutions for optically thin media for arbitrary source distributions. However it does not capture shadowing (unlike M1) and is not manifestly photon-conserving. If you use these modules in scientific work, please cite Hopkins et al. 2012 (MNRAS 2012, 421, 3488) and Hopkins et al., 2017, arXiv:1702.06148.

Numerical Solvers & Optimization

############################################################################################################################
#--------------------- solvers (numerical) --------------------------------------------------------
#RT_SPEEDOFLIGHT_REDUCTION=1            # set to a number <1 to use the 'reduced speed of light' approximation for photon propagation (C_eff=C_true*RT_SPEEDOFLIGHT_REDUCTION)
#RT_COMOVING                            # solve RHD equations formulated in the comoving frame, as compared to the default mixed-frame formulation; see Mihalas+Mihalas 84
############################################################################################################################

This governs decisions of how the transport algorithm above is solved, for example whether the full speed of light is used or a reduced speed-of-light approximation.

RT_SPEEDOFLIGHT_REDUCTION: Set to a value below unity to enable the ‘reduced speed of light’ approximation, reduced by this factor

RT_COMOVING: By default, the code uses the more common mixed-frame approximation to RHD, where the RHD equations are solved in the lab/simulation frame, then boosted after solution/update into the appropriate translating-with-the-cells frame if one is using a Lagrangian solver (e.g. SPH, MFM, etc). By turning this module on, the code will instead solve the RHD equations directly formulated in the comoving frame, as compared to the default mixed-frame formulation. See Mihalas and Mihalas 1984 for detailed discussion of the differences. Formally these are equivalent to leading order in u/c, where u is the characteristic fluid velocity. The lab-frame system includes some slightly higher terms relevant at higher fluid velocities, but is also less accurate and less robust to very small reduced-speed of light if that is used with a comoving/Lagrangian formulation (note that none of the formulations included by default in GIZMO RHD right now is valid for fully-relativistic MHD plus RHD).

Frequencies/Wavebands Evolved

############################################################################################################################
# -------------------- physics: wavelengths+coupled RT-chemistry networks (if any of these is used, cite Hopkins et al. 2018, MNRAS, 480, 800) -----------------------------------
#RT_SOURCES=1+16+32                     # source types for radiation given by bitflag (1=2^0=gas,16=2^4=new stars,32=2^5=BH)
#RT_XRAY=3                              # x-rays: 1=soft (0.5-2 keV), 2=hard (>2 keV), 3=soft+hard; used for Compton-heating
#RT_CHEM_PHOTOION=2                     # ionizing photons: 1=H-only [single-band], 2=H+He [four-band]
#RT_LYMAN_WERNER                        # specific lyman-werner [narrow H2 dissociating] band
#RT_PHOTOELECTRIC                       # far-uv (8-13.6eV): track photo-electric heating photons + their dust interactions
#RT_NUV                                 # near-UV: 1550-3600 Angstrom (where direct stellar emission dominates)
#RT_OPTICAL_NIR                         # optical+near-ir: 3600 Angstrom-3 micron (where direct stellar emission dominates)
#RT_FREEFREE                            # scattering from Thompson, absorption+emission from free-free, appropriate for fully-ionized plasma
#RT_INFRARED                            # infrared: photons absorbed in other bands are down-graded to IR: IR radiation + dust + gas temperatures evolved independently. Requires METALS and COOLING.
#RT_GENERIC_USER_FREQ                   # example of an easily-customizable, grey or narrow band: modify this to add your own custom wavebands easily!
#RT_OPACITY_FROM_EXPLICIT_GRAINS        # calculate opacities back-and-forth from explicitly-resolved grain populations. Cite Hopkins et al., arXiv:2107.04608, if used.
############################################################################################################################

This governs the wavelengths and source functions to be solved in the code. Note that one can choose an arbitrary combination of source particle types and wavebands – the code is modular so any set of bands is perfectly valid. For any given band, of course, the source functions and opacities must actually exist. If you wish to add a band to the code, you can do so by adding the appropriate source function and opacity functions to the existing functions rt_get_source_luminosity and rt_kappa within the file rt_utilities.c. Follow the template that exists there now.

If you use any of these, please cite the relevant methods papers where these modules were presented and developed. A methods paper with the details for these modules is Hopkins et al., 2019, MNRAS, “Radiative Stellar Feedback in Galaxy Formation: Methods and Physics”. But you can also cite Hopkins et al. 2018, MNRAS, 480, 800.

RT_SOURCES: Particle types which can act as radiation sources. This should be set as a bitmask (see notes on gravity flags)

RT_XRAY: Follow RT for X-ray bands. Value determines which bands: 1=soft (0.5–2 keV), 2=hard (>2 keV), 3=soft+hard. If the standard COOLING modules are turned on, this will directly enter the cooling module calculations for Compton heating.

RT_CHEM_PHOTOION: Follows RT for H and He photo-ionizing energies (912 Angstrom through 0.5 keV). If set =1, then this is a single band (all ionizing radiation is lumped together, energies 13.6–500 eV). If set =2, then a four-band H+He set is followed, with bands 13.6–24.6, 24.6–54.4, 54.4–70, and 70–500 eV. This allows independent bands for Helium-ionizing and doubly-ionizing photons, and very hard photons. If the standard COOLING modules are turned on, these will directly enter the cooling and ionization state calculations, and full non-equilibrium ionization states for H and He will be evolved.

RT_LYMAN_WERNER: Follow RT for Lyman-Werner band (H2-dissociating) flux. Sources and opacities are implemented, but self-consistent chemistry linking this to the H2 fraction is not yet fully-developed. Partial implementation in standard cooling; this physics can easily be linked to the Grackle cooling libraries as well (development encouraged as needed on a problem-by-problem basis).

RT_PHOTOELECTRIC: Follow RT for a photo-electric heating band (far-UV from 8–13.6eV). If the standard COOLING modules are turned on, this will directly enter the cooling module calculations for the photo-electric heating rate (using the local dust properties and photo-electric band photon densities).

RT_NUV: Follow RT for a near-UV band (from 1550–3600 Angstrom), where direct stellar emission dominates.

RT_OPTICAL_NIR: Follow RT for an optical-NIR band (from 3600 Angstrom to 3 microns), where direct stellar emission dominates.

RT_FREEFREE: Follow RT for a broad-band (grey) treatment of scattering and free-free absorption and emission for a fully-ionized plasma. The scattering opacity here is isotropic Thompson scattering, while emission and absorption coefficients follow free-free gas scalings.

RT_INFRARED: Follow RT for IR bands. The IR band is treated slightly differently from the other bands; rather than follow many narrow bands to treat a detailed IR spectrum, the follow the IR as a quasi-blackbody spectrum with a single band. However, the IR radiation temperature, dust temperature, and gas temperatures are strictly de-coupled and evolved independently according to the correct equations (they can of course come into equilibrium in sufficiently dense environments). The opacities and source functions account for this, and for the local dust abundance (if dust tracking is not explicit, this is scaled to a constant dust-to-metals ratio). Photons absorbed in other bands except for the ionizing bands (or others, if the user specifies by hand in the code) are assumed to down-grade into the IR and be re-emitted there. The explicitly solved dust temperatures and IR radiation temperatures are used in the cooling functions if the standard COOLING module is enabled. Because of this detailed modeling, this module requires both METALS and COOLING to function fully as intended.

RT_GENERIC_USER_FREQ: This is a handy convenience flag. Turning it on enables an easily-customizable grey or narrow-band for RHD. Search the code for the flag and you’ll see the couple places where you need to specify something, like the source injection rate, and opacities, but otherwise everything will work automatically. Modify this to add your own custom wavebands easily.

RT_OPACITY_FROM_EXPLICIT_GRAINS: Calculate opacities back-and-forth from explicitly-resolved grain populations. Specifically, with this flag on, the code will assume that the opacities to different wavelengths are dominated by dust, and use the explicitly-evolved populations of dust grains in the code to calculate those opacities at every position, in order to do the radiation transport, while simultaneously interpolating the radiation to the positions of the grains for calculation of quantities like their radiative accelerations. This follows the methodology detailed and tested in Hopkins et al., arXiv:2107.04608 – the full radiation-dust-magnetohydrodynamics (RDMHD) simulations therein. This requires GRAIN_FLUID be active, as well as one of the explicit radiation-hydrodynamics methods where the radiation fields are evolved on the gas cell grid (it has only been seriously tested with RT_M1 used). You can use an arbitrary set of wavelengths with this: since the grain sizes are known, the code will default to assuming a geometric opacity for them, unless you specify how the absorption efficiency \(Q\) scales as a function of grain size and wavelength, which you can easily do in the function return_grain_extinction_efficiency_Q in grain_physics.c – that’s where different radiative physics should be included. You can also specify an arbitrary function to give the grain albedo as a function of grain size and wavelength in the function rt_absorb_frac_albedo in rt_utilities.c. By default, if you have this flag and RT_GENERIC_USER_FREQ active so you are using a custom band, it will take a constant albedo from the parameterfile parameter Grain_Absorbed_Fraction_vs_Total_Extinction. Otherwise it will default to the albedos specified for all the other bands in the code. Make sure to cite Hopkins et al., arXiv:2107.04608 for any use of this, and reach out to PFH if this is planned to be used for new projects since it is still a set of modules under active development.

Additional Options

############################################################################################################################
#--------------------- radiation pressure options -------------------------------------------------
#RT_DISABLE_RAD_PRESSURE                # turn off radiation pressure forces (included by default)
#RT_RAD_PRESSURE_OUTPUT                 # print radiation pressure to file (requires some extra variables to save it)
#RT_ENABLE_R15_GRADIENTFIX              # for moments [FLD/OTVET/M1]: enable the Rosdahl+ 2015 approximate 'fix' (off by default) for gradients under-estimating flux when under-resolved by replacing it with E_nu*c
############################################################################################################################

Flags governing the radiation pressure terms (photon momentum transfer to gas), which are included by default for any RT method

RT_DISABLE_RAD_PRESSURE: Disable radiation pressure terms

RT_RAD_PRESSURE_OUTPUT: Output radiation pressure terms incident on all gas particles to the simulation snapshots

RT_ENABLE_R15_GRADIENTFIX: This is relevant for radiation pressure if and only if one of the moments methods (M1/FLD/OTVET) is being used to evolve the radiation. If those methods are used, then if this is on (it is off by default), the radiation pressure force is calculated using the approximate fix from Rosdahl et al. 2015, MNRAS, 451, 34 (Appendix B), whereby the optically-thin flux (\({\bf F}\rightarrow e_{\nu}c\hat{\bf F}\)) is used to replace the explicitly-calculated \({\bf F}\), as the latter is smoothed over ~10 grid cells around any point sources owing to the fact that it depends on numerical sourcing of the relevant gradient terms. This being on is generally more accurate in simulations with lots of point sources where the initial single-scattering mean-free-paths are poorly-resolved (e.g. star and galaxy formation simulations). However, if there are not point sources or the mean-free-paths are well-resolved, turning this off will yield more accurate behavior with these methods.

############################################################################################################################
# ----------- alternative, test-problem, or special behavior options
## ----------------------------------------------------------------------------------------------------
#RT_SELFGRAVITY_OFF                     # turn off gravity: if using an RT method that needs the gravity tree (FIRE, OTVET), use this -instead- of SELFGRAVITY_OFF to safely turn off gravitational forces
#RT_USE_TREECOL_FOR_NH=6                # uses the TreeCol method to estimate effective optical depth using non-local information from the gravity tree; cite Clark, Glover & Klessen 2012 MNRAS 420 754. Value specifies the number of angular bins on the sky for ray-tracing column density.
#RT_INJECT_PHOTONS_DISCRETELY           # do photon injection in discrete packets, instead of sharing a continuous source function. works better with adaptive timestepping (default with GALSF)
#RT_USE_GRAVTREE_SAVE_RAD_FLUX          # save radiative fluxes incident on each cell if using RHD methods that propagate fluxes through the gravity tree when these wouldn't be saved by default
#RT_REPROCESS_INJECTED_PHOTONS          # re-process photon energy while doing the discrete injection operation conserving photon energy, put only the un-absorbed component of the current band into that band, putting the rest in its "donation" bin (ionizing->optical, all others->IR). This would happen anyway during the routine for resolved absorption, but this may more realistically handle situations where e.g. your dust destruction front is at totally unresolved scales and you don't want to spuriously ionize stuff on larger scales. Assume isotropic re-radiation, so inject only energy for the donated bin and not net flux/momentum. follows STARFORGE methods (Grudic+ arXiv:2010.11254) - cite this
#RT_SINK_ANGLEWEIGHT_PHOTON_INJECTION     # uses a solid-angle as opposed to simple kernel weight (requires extra passes) for depositing radiation from sinks/BHs when the direct deposition is used. also ensures the sink uses a 2-way search to ensure overlapping diffuse gas gets radiation. cite Grudic+ arXiv:2010.11254
#RT_ISRF_BACKGROUND=1                   # include Draine 1978 ISRF for photoelectric heating (appropriate for solar circle, must be re-scaled for different environments); normalized by the parameter InterstellarRadiationFieldStrength (the value here is only an on/off switch), if defined
## ----------------------------------------------------------------------------------------------------
# ----------- Transport subcycling: allow RT and/or CR transport to take multiple smaller steps per hydro step  [HIGHLY EXPERIMENTAL]
## ----------------------------------------------------------------------------------------------------
#TRANSPORT_SUBCYCLE=100                 # max number of transport (RT+CR) subcycles per hydro step. decouples the transport CFL from the hydro timestep.
#TRANSPORT_SUBCYCLE_COOLING             # also subcycle cooling within the transport subcycle loop (otherwise cooling runs once per hydro step)
############################################################################################################################

RT_SELFGRAVITY_OFF: Turn off self-gravity without turning off RHD. If you are using an RT method that needs the gravity tree (e.g. LEBRON or OTVET), use this -instead- of SELFGRAVITY_OFF to safely turn off gravitational forces: otherwise the code will skip the gravity tree, and those methods will not work correctly.

RT_USE_TREECOL_FOR_NH: Uses the TreeCol method to estimate effective optical depth using non-local information from the gravity tree; cite Clark, Glover & Klessen 2012 MNRAS 420 754. Value specifies the number of angular bins on the sky for ray-tracing column density.

RT_INJECT_PHOTONS_DISCRETELY: Do photon injection in discrete packets, instead of sharing a continuous source function. This works better with adaptive timestepping, if timestep hierarchies are deep, because it can allow for arbitrary injection for non-gas cells (e.g. stars, black holes) which have arbitrary timestepping relative to the gas. As a result simulations of galaxies and stars (enabling GALSF) will automatically turn this option on by default.

RT_USE_GRAVTREE_SAVE_RAD_FLUX: Save radiative fluxes incident on each cell (and output them to snapshots) if using RHD methods that propagate fluxes through the gravity tree, when these wouldn’t be saved by default.

RT_REPROCESS_INJECTED_PHOTONS: Re-process photon energy while doing the discrete injection operation conserving photon energy. This will put only the un-absorbed component of the current band into that band, putting the rest in its donation bin (ionizing photons will be converted to optical, all others to IR). This would happen anyway during the routine for resolved absorption, but this may more realistically handle situations where e.g. your dust destruction front is at totally unresolved scales and you don’t want to spuriously ionize material on larger scales. Assume isotropic re-radiation, so inject only energy for the donated bin and not net flux/momentum. cite Grudic et al. arXiv:2010.11254 for this method.

RT_SINK_ANGLEWEIGHT_PHOTON_INJECTION: This uses a solid-angle as opposed to simple kernel weight (requires extra passes) for depositing radiation from sinks/BHs when the direct deposition is used. also ensures the sink uses a 2-way search to ensure overlapping diffuse gas gets radiation. cite Grudic+ arXiv:2010.11254

RT_ISRF_BACKGROUND: Enabling this will include Draine 1978 ISRF for photoelectric heating (appropriate for the solar circle, must be re-scaled for different environments); the normalization of this field is set by the run-time parameter InterstellarRadiationFieldStrength (the value given to this flag only acts as a switch). Otherwise in simulations with GALSF and COOLING just a meta-galactic UVB will be the uniform background. Enable for simulations where you want a galactic background but dont actually simulate galactic stellar populations that can source this.

TRANSPORT_SUBCYCLE: Lets the explicit radiation transport and cosmic-ray transport take several smaller sub-steps within each hydro step, so the transport Courant condition (often set by a large reduced speed of light) no longer sets the gas timestep. The value is the maximum number of sub-steps per hydro step. Each cell’s hydro timestep is limited to (value - 0.5) times its RT timestep, or value times its CR transport timestep, instead of to the transport timestep itself. On each step the code uses \(N = {\rm MIN}(\lceil \Delta t_{\rm hydro,max}/\Delta t_{\rm transport,min} \rceil,\ {\rm value})\) sub-steps, taken over the active cells across all tasks. On every sub-step it recomputes the transport fluxes and kicks the radiation and CR fields (and updates opacities) on a neighbor topology frozen for that hydro step. Source injection and the other non-flux RT operations still run once per hydro step. This replaces the retired RT_DIFFUSION_IMPLICIT (use RT_M1 with this instead). It forces the neighbor-list (“Mode A”) hydro path on every step. Written by Mike Grudic. This is highly experimental: please contact the developer before using it for production runs.

TRANSPORT_SUBCYCLE_COOLING: With TRANSPORT_SUBCYCLE, also calls the cooling/thermochemistry routine on every transport sub-step (each with \(1/N\) of the hydro timestep), instead of once per hydro step after the sub-cycle loop. This keeps the gas temperature, ionization state and dust-radiation exchange coupled to the radiation field as it evolves within the step, which can matter when the thermochemistry responds on the transport timescale. It costs \(N\) cooling calls per hydro step. Same development status and contact as TRANSPORT_SUBCYCLE.

Multi-Threading (Parallelization) Options

####################################################################################################
# --------------------------------------- Multi-Threading and Parallelization options
####################################################################################################
#OPENMP=2                       # top-level switch for explicit OpenMP implementation
#DOMAIN_SEGMENTS_SCALE=1        # scale the number of separate Peano-Hilbert segments each rank is given, relative to the automatic choice. The code picks that number at every full domain decomposition so each segment holds roughly ten thousand particles, which is where finer load-balancing stops paying for the extra top-tree refinement, pseudo-particle communication and spatial fragmentation it costs. Raise it for problems whose cost is dominated by load imbalance, lower it for problems limited by memory or communication.
#DOMAIN_TOPTREE_REFINEMENT_SCALE=1 # scale how finely the top tree is refined, relative to the number of domain segments it has to fill. Values below 1 are clamped away, since a top tree with fewer leaves than segments cannot be assigned. The effect on run time is much weaker than DOMAIN_SEGMENTS_SCALE.
#DOMAIN_TIMEBINS=0              # Domain timebin cost weighting: 0=frequency-weighted costs, 1=full per-timebin balancing (Gadget-4 scheme). Omit for unweighted.
#DOMAIN_NO_LIGHTWEIGHT_REPARTITION # force a full domain decomposition every time one is triggered. By default the code instead rebalances the load while reusing the existing top-level tree whenever a full decomposition was not actually required, which is much cheaper. Only set this if a run needs the top tree rebuilt every time.
#GPU_PARTICLE_STORAGE_PLACEMENT=1 # override where the particle storage prefers to live on an AMD/HIP GPU. The particle arrays are demand-paged between host and device; by default the code picks from the per-rank size of that storage, asking above roughly a gigabyte for the pages to stay on the host with the device mapped in as an accessor. THAT DEFAULT IS A STOPGAP that keeps the largest runs completing, and it works against the port: host-resident particle storage makes every routine moved onto the device reach across the fabric for it, so a device flip priced against it loses alone even when the same flips would win together. 0 = no hint, pages settle wherever they are used; 1 = prefer host, device mapped in; 2 = prefer device, host mapped in -- use 2 to measure a device flip against storage that moved with it. No effect on NVIDIA GPUs or on CPU-only builds.
#GPU_TREE_STORAGE_PLACEMENT=2 # override where the gravity tree's device mirror prefers to live on an AMD/HIP GPU. The mirror is the structure the device gravity walk reads -- it exists for that reason, and the walk reads nothing else, the tree's host-side representation being untouched by device code. Like the particle storage it is demand-paged, but unlike it the code asks for nothing by default, and no hint is not a neutral state: managed pages settle where they are FIRST touched, the host builds the tree, so the device then chases pointers through host-resident pages across the fabric for the whole of every walk. Which side should own it is genuinely moving -- the moment refresh and the node drift sweep are already device kernels writing this mirror, while the build and the lazy per-node repair still write it from the host -- so it is a knob rather than a constant. 0 = no hint (the default, i.e. today's behaviour, and a placeholder rather than a measured choice); 1 = prefer host, device mapped in; 2 = prefer device, host mapped in. No effect on NVIDIA GPUs or on CPU-only builds.
####################################################################################################

These flags govern multi-threading and domain subdivision in the code. Note that the bulk of the per-step compute (hydro, gravity, cooling, feedback, and so on) now runs through the Kokkos device backend — see the Performance-Portable Heterogeneous Execution section. Kokkos provides its own internal threading (its OpenMP backend on CPU-only nodes; CUDA/HIP on GPU). The OPENMP flag below controls only the residual host-side #pragma omp parallelization in code paths that have not been Kokkos-ized (mainly some I/O, MPI-staging, and a handful of bookkeeping loops). On a GPU build, OPENMP is largely irrelevant; on a CPU-only Kokkos build it is generally simpler to leave host OpenMP off and let Kokkos manage all the threads.

OPENMP: This enables the code to run in hybrid OpenMP-MPI (multi-threaded) mode for the host-side code paths. If you enable this, you need to edit the makefile compile command appropriately to be sure you are actually compiling with the required flags for OPENMP (on many machines the code will still compile and appear to be working normally, but you wont actually be multi-threading!). For example, on TACC machines like Frontera, “-qopenmp” is added to the intel compiler OPT line. Set the value of OPENMP equal to the number of threads per MPI process. This can be anything from 1 (which is not useful, so make it at least 2 if you enable it at all) to the number of MPI cores on the node (some systems allow more threads than cores, but you should read up on your machine before you attempt anything like that). In the submission script for the job, you will also usually need to specify a flag for the number of threads you are using – again, consult the guide for your local machine for running in hybrid OpenMP+MPI mode. The OpenMP threads use shared memory, so they must be on the same node; but only certain parts of the code can be multi-threaded. In general, this option is here to both allow larger parallelization and better code scaling for large simulations, and to help deal with memory errors (since OpenMP is shared memory, two threads on two cores will have effectively double the memory of two MPI processes on two cores). However, the gains (or losses) are highly dependent on both the specific problem, resolution, processor number, and machine configuration. You should experiment with this setting to get an idea of which values may be useful.

DOMAIN_SEGMENTS_SCALE: The domain decomposition hands each MPI task its share of the simulation volume as several separate contiguous pieces of the Peano-Hilbert curve, rather than one. Having more pieces makes the decomposition far less dependent on spatial co-location and lets it balance the work much more finely, at the cost of more top-level tree refinement, more communication (each piece exchanges its own pseudo-particle data), and a more spatially fragmented territory per task. The code chooses the number of pieces automatically from the particle count and the number of tasks, aiming for roughly ten thousand particles per piece. It is set when the initial conditions are read and revisited at each full domain decomposition, so a run whose particle count grows keeps a granularity matched to its load; the value is rounded to a power of two, so in practice it only moves when the load genuinely doubles. That target is measured to be about where the balancing gain stops paying for those costs, and it holds across problems spanning more than an order of magnitude in size. This setting multiplies that automatic choice, so the default of 1 leaves it alone; the result is rounded to a power of two and capped, so raising it has no further effect once the cap is reached. Raise it for problems whose cost is dominated by load imbalance; lower it for problems that are memory- or communication-limited. Note the number of simultaneous communication requests is the task count times the number of pieces per task, which is a common source of trouble if it is pushed very high.

DOMAIN_TOPTREE_REFINEMENT_SCALE: Multiplies how finely the top-level tree is refined, relative to the number of domain pieces that refinement has to fill. The default of 1 asks for one leaf per piece; refinement splits eight ways at a time, so what is achieved overshoots what is asked for, and both numbers are reported at each decomposition. Values below 1 are clamped, because a top tree with fewer leaves than pieces cannot be assigned at all. This has a much weaker effect on run time than DOMAIN_SEGMENTS_SCALE, and is rarely worth changing.

DOMAIN_TIMEBINS: Chooses how the cost of each domain is weighted when the decomposition balances the load. Set =0 to weight by how frequently each element is active, or =1 for full per-timebin balancing following the Gadget–4 scheme. Omit the flag entirely to leave the costs unweighted.

DOMAIN_NO_LIGHTWEIGHT_REPARTITION: Forces a full domain decomposition every time one is triggered. By default the code will instead re-balance the load while re-using the existing top-level tree, whenever a full decomposition was not actually required. That is much cheaper, and on large runs – most of all on GPUs, where a full decomposition is disproportionately expensive – the difference is substantial. Set this only if a run genuinely needs the top-level tree rebuilt every time.

GPU_PARTICLE_STORAGE_PLACEMENT: Overrides where the particle storage prefers to live on an AMD/HIP GPU. These arrays are demand-paged between host and device, so each page ends up wherever it was last touched, and by default the code chooses a placement from the per-rank size of that storage: above roughly a gigabyte it asks for the pages to prefer host memory with the device mapped in as an accessor, and below it leaves them to settle wherever they are used. The size dependence is the point – on a small problem the repeated device walks give each page enough re-use that it settles in device memory and stays there, while on a large one nothing gets that re-use, host and device each sweep the whole array every cycle, and the migration never converges. The difference is worth a factor of several on the largest runs, and a few percent the other way on small ones. Set =1 to force the host-preferred placement or =0 to force the default, for instance to measure the crossover on a problem near it. Set =2 to prefer device memory with the host mapped in, for instance to measure a routine moved onto the device against particle storage that moved with it. This has no effect on NVIDIA GPUs, where the equivalent advice has not been tested, or on CPU-only builds.

GPU_TREE_STORAGE_PLACEMENT: Overrides where the device copy of the gravity tree prefers to live on an AMD/HIP GPU. This is the copy that the device gravity walk reads; the host walk uses its own representation, which is unaffected. Like the particle storage it is demand-paged between host and device. Unlike the particle storage, the code gives it no placement hint by default. An unhinted page settles where it is first touched, and the host builds the tree, so by default the mirror is host-resident and the device reads it across the host-device link on every walk. Set =1 to prefer host memory with the device mapped in, or =2 to prefer device memory with the host mapped in. =0 (the default) gives no hint. The default is a placeholder rather than a measured optimum: some writes to this mirror already happen on the device (moment refresh, node drift) while the build and per-node repairs still happen on the host, so the best choice can depend on the problem and code version. Whether the hint was accepted is printed once at startup. No effect on NVIDIA GPUs or on CPU-only builds.

​

Input/Output Options

These flags govern snapshot outputs (what is saved and how it is saved).

####################################################################################################
# --------------------------------------- Input/Output options
####################################################################################################
#OUTPUT_ADDITIONAL_RUNINFO      # enables extended simulation output data (can slow down machines significantly in massively-parallel runs)
#OUTPUT_IN_DOUBLEPRECISION      # snapshot files will be written in double precision
#INPUT_IN_DOUBLEPRECISION       # input files assumed to be in double precision (otherwise float is assumed)
#INPUT_POSITIONS_IN_DOUBLE      # as above, but specific to the ICs file
#OUTPUT_POTENTIAL               # forces code to compute+output potentials in snapshots
#OUTPUT_TIDAL_TENSOR            # writes tidal tensor (computed in gravity) to snapshots
#OUTPUT_ACCELERATION            # output physical acceleration of each particle in snapshots
#OUTPUT_CHANGEOFENERGY          # outputs rate-of-change of internal energy of gas particles in snapshots
#OUTPUT_VORTICITY               # outputs the vorticity vector
#OUTPUT_BFIELD_DIVCLEAN_INFO    # outputs the phi, phi-gradient, and numerical div-B fields used for de-bugging MHD simulations
#OUTPUT_TIMESTEP                # outputs timesteps for each particle
#AGS_OUTPUTZETA                 # outputs the adaptive-softening correction term zeta for each adaptively-softened particle
#OUTPUT_COOLRATE                # outputs the net cooling rate (negative where gas is net heating), and conduction rate if enabled
#OUTPUT_COOLRATE_DETAIL         # outputs cooling rate term by term [saves all individually to snapshot]
#OUTPUT_POWERSPEC               # compute and output cosmological power spectra. requires BOX_PERIODIC and PMGRID.
#OUTPUT_RECOMPUTE_POTENTIAL     # update potential every output even it EVALPOTENTIAL is set
#OUTPUT_DENS_AROUND_NONGAS        # output gas density in neighborhood of stars [collisionless particle types], not just gas
#OUTPUT_DELAY_TIME_HII          # output DelayTimeHII. Requires GALSF_FB_FIRE_RT_HIIHEATING (and corresponding flags set)
#OUTPUT_MOLECULAR_FRACTION      # output the code-estimated molecular mass fraction [needs COOLING], for e.g. approximate molecular fraction estimators (as opposed to detailed chemistry modules, which already output this)
#OUTPUT_TEMPERATURE             # output the in-code gas temperature
#OUTPUT_SINK_ACCRETION_HIST     # save full accretion histories of sink (BH/star/etc) particles
#OUTPUT_SINK_FORMATION_PROPS    # save at-formation properties of sink particles
#OUTPUT_SINK_DISTANCES          # saves the distance to the nearest sink, if SINK_CALC_DISTANCES is enabled, to snapshots
#OUTPUT_RT_RAD_FLUX             # save flux vector for radiation methods that explictly evolve the flux (e.g. M1)
#OUTPUT_TURB_DIFF_DYNAMIC_ERROR # save error terms from localized dynamic Smagorinsky model to snapshots
#OUTPUT_TWOPOINT_ENABLED        # allows user to calculate mass 2-point function by enabling and setting restartflag=5
#INPUT_READ_KERNELRADIUS        # force reading rkern from IC file (instead of re-computing them; in general this is redundant but useful if special guesses needed)
#INPUT_READ_SINKPROPS           # force reading sink properties including sink radius, zams mass, luminosity, age, etc, from ICs file if it includes sink particles and the IC is designed for use with the single-star modules
#IO_COMPRESS_HDF5     		    # write HDF5 in compressed form (will slow down snapshot I/O and may cause issues on old machines, but reduce snapshots 2x)
#IO_SUPPRESS_TIMEBIN_STDOUT=10  # only prints timebin-list to log file if highest active timebin index is within N (value set) of the highest timebin (dt_bin=2^(-N)*dt_bin,max)
#IO_SUBFIND_READFOF_FROMIC      # try read already existing FOF files associated with a run instead of recomputing them: not de-bugged
#IO_MOLECFRAC_NOT_IN_ICFILE     # special flag needed if using certain molecular modules with restart flag=2 where molecular data was not in that snapshot, to tell code not to read it
#IO_REPAIR_COINCIDENT_POSITIONS # opt-in repair of invalid input data in which two particles share a position to the bit. By default such data stops the run, since it is an invalid IC/snapshot and should be seen rather than silently altered; this instead separates each pair once at startup, by 1e-4 of the local interparticle spacing along an ID-seeded direction, preserving the pair's center of mass, and logs every repaired ID and position. Leave off unless deliberately repairing a known-bad input file.
#IO_REDUNDANT_BACKUP_RESTARTFILE_FREQUENCY=3  # keep an extra set of backup files that are REDUNDANT_BACKUP_RESTARTFILE_FREQUENCY number of restarts old (allows for soft restarts from an older position)
#IO_GRADUAL_SNAPSHOT_RESTART       # when restarting from a snapshot (flag=2) start every element on the shortest possible timestep - can reduce certain transient behaviors from the restart procedure
#IO_SINKS_ONLY_SNAPSHOT_FREQUENCY # determines the number of snapshots with reduced data (sinks only) per full snapshots (gas+sinks+other), e.g., setting this to 2 means 2/3 of the snapshots will be reduced, 1/3 will have full data. Setting this to 0 disables it. developed by DG.
####################################################################################################

IO_DISABLE_HDF5: If this is set, the code will be compiled without support for input and output in the HDF5 format. You need to have the HDF5 libraries and headers installed on your computer for the code to compile otherwise (which you should do). The HDF5 format format is normally format “3” in the parameterfile. Without it, the code will resort to the un-formatted fortran binary as the only option (format “1”), which is not recommended.

OUTPUT_ADDITIONAL_RUNINFO: This enables various outputs that are disabled by default. This includes outputs like ‘cpu.txt’ every timestep instead of only on the super-steps, far more output in the stdout outputs, checking for ‘stop’ files for checkpointing every timestep, outputing the extended sink_details.txt files, etc. On machines with slow I/O, this can be a major slow-down and bottleneck for the code. On machines with fast I/O, this is much less of an issue. It depends strongly on the run. But most of the extra output is only needed for de-bugging.

OUTPUT_IN_DOUBLEPRECISION: Snapshots written in double-precision (single is default, even though the code uses double; this is purely to save on storage).

INPUT_IN_DOUBLEPRECISION: Tells to code to assume the IC file is in double-precision. Otherwise it will assume single-precision (if you get this wrong, youll read-in nonsense values and the code will crash).

OUTPUT_POSITIONS_IN_DOUBLE: Positions/coordinates will specifically be output in double, even if the rest of the snapshot is output in floating point precision. This is useful for very large simulations, where the length units might be large (say, Mpc), and the resolution small (say, pc-scale), so the difference in positions between particles requires double precision to capture.

INPUT_POSITIONS_IN_DOUBLE: Tells the code to assume the positions in the input file are in double precision (even if the rest of the file is not).

OUTPUT_RECOMPUTE_POTENTIAL: Re-computes the potential energy of particles, every time an output is written, so that the potential is accurate to the current time (as opposed to drifted from the earlier computation: since potential is not directly used by the code, it is not evaluated at all timesteps). This can add a non-negligible expense to the code.

The remaining flags in this section all turn on/off additional (optional) outputs to snapshots, which can often be quite useful (but are by default excluded to save on storage requirements). Many other physics modules have additional such output flags hard-wired to output certain quantities used in the module. If youre curious, you should look at the file “io.c” – under e.g. “get_dataset_name” you can see a list of the names of the different datasets which can be output. Here are some of the more common output flags:

OUTPUT_POTENTIAL: Add potential energy to snapshots (under HDF5 buffer header name “Potential”)

#OUTPUT_TIDAL_TENSOR: Add tidal tensor (computed in gravity) to snapshots (HDF5 “TidalTensor”)

OUTPUT_ACCELERATION: Add acceleration to snapshots (HDF5 “Acceleration”)

OUTPUT_CHANGEOFENERGY: Add time derivative of internal energy to snapshots (HDF5 “RateOfChangeOfInternalEnergy”)

OUTPUT_VORTICITY: Add local vorticity to snapshots (HDF5 “Vorticity”)

OUTPUT_BFIELD_DIVCLEAN_INFO Adds values of the quantities used for the magnetic field cleaning procedure with the default Dedner algorithm to snapshots. Specifically this outputs the phi, phi-gradient, and numerical div-B fields used for de-bugging MHD simulations

OUTPUT_TIMESTEP: Add particle timestep dt to snapshots (HDF5 “TimeStep”)

OUTPUT_COOLRATE: Add particle cooling rate to snapshots (HDF5 “CoolingRate”). Enabling OUTPUT_COOLRATE_DETAIL in addition produces many additional details.

OUTPUT_COOLRATE_DETAIL: Outputs cooling rate term by term (saves all individually to snapshot) – this separates things like metal-line cooling, molecular, dust, hydrodynamic work, etc, useful for detailed modeling and diagnosis.

OUTPUT_POWERSPEC: Compute and output cosmological matter power spectra. Requires BOX_PERIODIC and PMGRID.

OUTPUT_DENS_AROUND_NONGAS: Outputs the gas density in the neighborhood of each non-gas particle (all types 1–5, e.g. stars and sinks), evaluated from its gas neighbors (HDF5 “DensityAtParticleLocation”). Enabling this turns on the neighbor-density calculation for non-gas particles if it is not already on (it is automatically on with GALSF, SINK_PARTICLES, RADTRANSFER, CHIMES or RT_REPROCESS_INJECTED_PHOTONS).

OUTPUT_DELAY_TIME_HII: Outputs the DelayTimeHII variable representing the remaining time for which a gas cell next to a star is flagged as part of a compact HII region, according to the GALSF_FB_FIRE_RT_HIIHEATING module.

OUTPUT_MOLECULAR_FRACTION: Output the code-estimated molecular mass fraction (needs COOLING), for e.g. approximate molecular fraction estimators (as opposed to detailed chemistry modules, which already output this).

OUTPUT_TEMPERATURE: Output the in-code gas temperature. Useful when complicated equations-of-state are involved, so the internal energy (normally output) is not sufficient to calculate this.

OUTPUT_GRADIENT_RHO: Output the density gradient vector for each gas particle (HDF5 “DensityGradient”). Useful for post-processing analysis of shocks, interfaces, and sub-grid models that depend on local density gradients.

OUTPUT_GRADIENT_VEL: Output the velocity gradient tensor for each gas particle (HDF5 “VelocityGradient”). The full 3x3 tensor is saved, which can be used to compute vorticity, strain rate, divergence, and other derived quantities in post-processing.

OUTPUT_HYDROACCELERATION: Output the hydrodynamic acceleration (from the hydro force solver, excluding gravity) for each gas particle (HDF5 “HydroAcceleration”). Useful for diagnosing the relative importance of pressure, magnetic, and gravitational forces.

OUTPUT_RT_RAD_OPACITY: Output the radiation opacity for each particle and frequency bin (HDF5 “PhotonOpacity”). Requires one of the radiation transport methods to be enabled.

OUTPUT_SHOCK_MACH_NUMBER: Output the estimated Mach number of the strongest shock passing through each gas particle (HDF5 “ShockMachNumber”). The Mach number is estimated from the Rankine-Hugoniot jump conditions applied to the Riemann solver states at particle interfaces. Useful for shock-finding and cosmic ray injection at shocks.

OUTPUT_SOFTENING: Output the gravitational softening length for each particle (HDF5 “Softening_KernelRadius”). Particularly useful when adaptive gravitational softening is enabled (ADAPTIVE_GRAVSOFT_FORGAS or ADAPTIVE_GRAVSOFT_FORALL), where softenings vary per particle.

AGS_OUTPUTZETA: Output the correction term \(\zeta\) of the adaptive gravitational softening for each adaptively-softened particle (HDF5 “AGS-Zeta”): the term that keeps the equations of motion energy- and momentum-conserving when the softening varies with position (Price & Monaghan 2007). Only meaningful when adaptive softenings are active (e.g. ADAPTIVE_GRAVSOFT_FORALL).

OUTPUT_UNSPAWNED_SINKMASS: Output the mass that has been flagged for spawning into a sink particle but has not yet been spawned (HDF5 “UnspawnedSinkMass”). Relevant for sink particle formation models where mass accumulation occurs over multiple timesteps before the sink is created.

OUTPUT_SINK_ACCRETION_HIST: Save full accretion histories of sink (BH/star/etc) particles.

OUTPUT_SINK_FORMATION_PROPS: Save various at-formation properties of sink particles.

OUTPUT_SINK_DISTANCES: Saves the distance to the nearest sink, if SINK_CALC_DISTANCES is enabled, to snapshots.

OUTPUT_RT_RAD_FLUX: Save flux vector for radiation methods that explictly evolve the flux (e.g. M1).

OUTPUT_TURB_DIFF_DYNAMIC_ERROR : Save error terms from localized dynamic Smagorinsky model to snapshots.

There are a few special flags governing special behaviors on snapshot startup or restart and output.

INPUT_READ_KERNELRADIUS: Read the initial guess for KernelRadius from the ICs file. In any case the density routine must be called to determine the “correct” KernelRadius, but this can be useful if the ICs are irregular so the “guess” the code would normally use to begin the iteration process might be problematic. In general though it is redundant.

INPUT_READ_SINKPROPS: Forces the sink properties – sink radius, ZAMS mass, luminosity, age and the rest – to be read from the initial conditions file, for an IC which contains sink particles and was built for use with the single-star modules. Without this those properties are initialized by the code rather than taken from the file.

OUTPUT_TWOPOINT_ENABLED: Allows the user to calculate mass 2-point function by enabling and setting restartflag=5, and then running the code with this enabled. It will calculate the correlation function and immediately write a snapshot, then cease.

IO_SUBFIND_READFOF_FROMIC: This tells to code to try read already existing FOF files associated with a run instead of recomputing them.

IO_MOLECFRAC_NOT_IN_ICFILE: This is a special flag needed if using certain molecular modules with restart flag=2 where molecular data was not in that snapshot, to tell code not to read it.

IO_REPAIR_COINCIDENT_POSITIONS: On the first domain ordering the code checks the input for pairs of particles at the same position. This includes pairs that are not identical to the bit but sit far below the local interparticle spacing, which roundoff will collapse onto each other within a few hundred steps. Such data has no relative geometry for the neighbor search or tree, so by default the run stops and lists the offending pairs. With this flag the code instead separates each pair once at startup, by \(10^{-4}\) of the local scale (the distance to the nearest other particle, or the softening if smaller), along a pseudo-random direction restricted to the active dimensions. The pair’s center of mass is preserved, and every repaired ID and its old and new positions are printed. Pairs that cannot be separated by an amount both above coordinate roundoff and well below the local scale are still fatal. Leave this off unless you are deliberately repairing a known-bad IC or snapshot file.

IO_GRADUAL_SNAPSHOT_RESTART: When restarting from a snapshot (flag=2) start every element on the shortest possible timestep - can reduce certain transient behaviors from the restart procedure

IO_SINKS_ONLY_SNAPSHOT_FREQUENCY: Outputs some snapshots with a reduced data set including only sink particles. Determines the number of snapshots with reduced data (sinks only) per full snapshots (gas+sinks+other), e.g., setting this to 2 means 2/3 of the snapshots will be reduced, 1/3 will have full data. Setting this to 0 disables it. Developed by David Guszejnov.

And some additional format options:

IO_COMPRESS_HDF5: This will attempt to write HDF5 in compressed form (will slow down snapshot I/O and may cause issues on old machines, but reduce snapshot sizes by a factor of about 2). Note due to some inefficiencies in the built-in HDF5 compression algorithm, this can take enormous time on massive-parallel runs, in which case it is far more efficient to compress the snapshots in serial after they are output with a separate script.

IO_SUPPRESS_TIMEBIN_STDOUT: Only prints timebin-list to log file if highest active timebin index is within \(N\) (value set) of the highest timebin (\(dt_{bin}=2^{-N} dt_{bin,max}\)).

IO_REDUNDANT_BACKUP_RESTARTFILE_FREQUENCY: Keep an extra set of backup files that are this number of restarts old (allows for soft restarts from an older position)

De-Bugging and Special Code Behaviors

These are miscellaneous flags for de-bugging and special purpose behaviors. If you don’t know what a flag here does, you should not change it. If you do, you should still treat the changes carefully and be sure it is ok for your particular problem.

####################################################################################################
# -------------------------------------------- De-Bugging & special (usually test-problem only) behaviors
####################################################################################################
# --------------------
# ----- General De-Bugging and Special Behaviors
#DEVELOPER_MODE                 # allows you to modify various numerical parameters (courant factor, etc) at run-time
#FORCE_EQUAL_TIMESTEPS          # force the code to use a single universal timestep (can change in time, but all particles advance together). chosen as minimum of any particle that step.
#STOP_WHEN_BELOW_MINTIMESTEP    # forces code to quit when stepsize wants to go below MinSizeTimestep specified in the parameterfile
#CONTINUE_BELOW_MINTIMESTEP        # opts out of the above, clamping the stepsize to MinSizeTimestep and carrying on. only for a problem where reaching the floor is expected and survivable
# --------------------
# ----- Hydrodynamics
#FREEZE_HYDRO                   # zeros all fluxes from RP and doesn't let particles move (for testing additional physics layers)
#EOS_ENFORCE_ADIABAT=(1.0)      # if set, this forces gas to lie -exactly- along the adiabat P=EOS_ENFORCE_ADIABAT*(rho^GAMMA)
#HYDRO_REPLACE_RIEMANN_KT       # replaces the hydro Riemann solver (HLLC) with a Kurganov-Tadmor flux derived in Panuelos, Wadsley, and Kevlahan, 2019. works with MFM/MFV/fixed-grid methods. more diffusive, but smoother, and more stable convergence results
#SLOPE_LIMITER_TOLERANCE=1      # sets the slope-limiters used. higher=more aggressive (less diffusive, but less stable). 1=default. 0=conservative. use on problems where sharp density contrasts in poor particle arrangement may cause errors. 2=original (more aggressive) slope limiters
#WAKEUP=4.1                        # timestep-limiter factor (Saitoh & Makino 2009): a cell is woken when a neighbour's timestep is this much shorter than its own. smaller is more accurate and more expensive (their Table 1: f=2 gives a smaller energy error than f=4 on Sedov, at roughly twice the cost). if unset, follows SLOPE_LIMITER_TOLERANCE: 4.1 when that is >0, else 2.1
#ENERGY_ENTROPY_SWITCH_IS_ACTIVE # enable energy-entropy switch as described in GIZMO methods paper. This can greatly improve performance on some problems where the
                                # the flow is very cold and highly super-sonic. it can cause problems in multi-phase flows with strong cooling, though, and is not compatible with non-barytropic equations of state
#FORCE_ENTROPIC_EOS_BELOW=(0.01) # set (manually) the alternative energy-entropy switch which is enabled by default in MFM/MFV: if relative velocities are below this threshold, it uses the entropic EOS
#HYDRO_KERNEL_SURFACE_VOLCORR      # attempt to correct SPH/MFM/MFV cell volumes for free-surface effects, using the estimated boundary correction for the Wendland C2 kernel (works with others but most accurate for this) based on asymmetry of neighbors within kernel, as calibrated in Reinhardt & Stadel 2017 (arXiv:1701.08296), see e.g. their Fig 3
#DISABLE_SURFACE_VOLCORR        # disables HYDRO_KERNEL_SURFACE_VOLCORR if it would be set by default (e.g. if EOS_ELASTIC is enabled)
#HYDRO_EXPLICITLY_INTEGRATE_VOLUME # explicitly integrate the kernel continuity equation for cell volumes (giving e.g. densities), as in e.g. Monaghan 2000, but with a term that relaxes the integrated cell volume back to the explicitly evaluated kernel calculation on a timescale ~10 t_cross where t_cross ~ MAX(H_kernel , L_grad) / MIN(cs_eff) where L_grad is the density gradient scale length and cs_eff the minimum sound/torsion/tension wave speed. This module ONLY makes sense for strictly fixed-mass (SPH/MFM) methods
#DISABLE_EXPLICIT_VOLUME_INTEGRATION # disables HYDRO_EXPLICITLY_INTEGRATE_VOLUME if it would be set by default (e.g. if EOS_ELASTIC is enabled)
# --------------------
# ----- Additional Fluid Physics and Gravity
#COOLING_OPERATOR_SPLIT         # do the hydro heating/cooling in operator-split fashion from chemical/radiative. slightly more accurate when tcool >> tdyn, but much noisier when tcool << tdyn
#COOL_LOWTEMP_THIN_ONLY         # in the COOL_LOW_TEMPERATURES module, neglect the suppression of cooling at very high surface densities due to the opacity limit (disables limiter in Eqs B29-B30, Hopins et al arXiv:1702.06148)
#MHD_ALTERNATIVE_LEAPFROG_SCHEME # use alternative leapfrog where magnetic fields are treated like potential/positions (per Federico Stasyszyn's suggestion): still testing
#SUPER_TIMESTEP_DIFFUSION       # use super-timestepping to accelerate integration of diffusion operators [for testing or if there are stability concerns]
#TURB_DRIVING_UPDATE_FORCE_ON_TURBUPDATE # if this is enabled, we only update as frequently as the driving phases are recomputed, as set by TurbDrive_TimeBetweenTurbUpdates. Only enable as an optimization if the cost of evaluating the turbulent force is large. To avoid large errors, TurbDrive_TimeBetweenTurbUpdates must be set by-hand to be << lambda_min / V where V is the typical turbulent velocity and lambda_min is the smallest driven wavelength.
#EVALPOTENTIAL                  # computes gravitational potential
#GRAVITY_HYBRID_OPENING_CRIT    # use -both- Barnes-Hut + relative angle opening criterion for the gravity tree (normally choose one or the other)
#TIDAL_TIMESTEP_CRITERION       # replace standard acceleration-based timestep criterion with one based on the tidal tensor norm, which is more accurate and adaptive (testing, but may be promoted to default code)
#ADAPTIVE_TREEFORCE_UPDATE=0.06 # use the tidal timescale to estimate how often gravity needs to be updated, updating a gas cell's gravity no more often than ADAPTIVE_TREEFORCE_UPDATE * dt_tidal, the factor N_f in Grudic 2020 arxiv:2010.13792 (cite this). Smaller is more accurate, larger is faster, should be tuned for your problem if used.
#MAINTAIN_TREE_IN_REARRANGE     # don't rebuild the domains/tree every time a particle is spawned - salvage the existing one by redirecting pointers as needed. cite Grudic+ arXiv:2010.11254
#RANDOMIZE_GRAVTREE             # move the top tree node around randomly so that treeforce errors are not correlated between one treebuild and another. cite Grudic+ arXiv:2010.11254 With periodic gravity the whole frame moves at each decomposition, so the setups anchored in space are refused at compile time: GRAVITY_ANALYTIC, GRAVITY_SPHERICAL_SYMMETRY, TURB_DRIVING, reflecting/outflow boundaries, SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM and FOF/SUBFIND.
#TREE_LEAF_BUCKET_SIZE=4        # stop subdividing tree nodes at this many elements, so a leaf holds several which the walk evaluates directly: smaller, shallower, cheaper-to-build and cheaper-to-communicate tree, traded against more direct element-element work per walk. Problem-dependent, worth tuning if you care about speed; 1 recovers one element per leaf.
#TREE_QUERY_PACKET_SIZE=8      # walk the gravity tree for this many adjacent active elements at once: they share one traversal while every element keeps its own opening decisions and evaluates its own accepted elements, so forces are unchanged and the traversal cost of a deep tree is divided among them. Default 8; problem-dependent, worth tuning if you care about speed; 1 walks every element alone. On the device the team walking a packet is generally WIDER than the packet: every thread in it traverses (so even a single element gets a whole team on its descent) while the first elements-many threads also evaluate. So this sets how many elements SHARE a traversal, not how many threads perform one; the count is additionally capped by the team the call runs on, so a larger value is walked as several packets and the members actually used are reported as Qdev. A device call whose elements reach about half the device's lanes walks one element per lane instead, since independent walks are then faster, so on the device this matters only below that (the host walk's packets are unaffected). The shape actually used is in the per-call record in timings.txt (packet: Q= T= Qdev= walkers= ...).
#GRAVITY_SPHERICAL_SYMMETRY=0   # modifies the tree gravity solver to give the solution assuming spherical symmetry about the origin (if BOX_PERIODIC is not enabled) or the box center. Useful for IC generation and test problems. Numerical value specifies a minimum softening length.  (cite Lane et al., arXiv:2110.14816)
#SINGLE_STAR_DIRECT_GRAVITY_RADIUS=1000. # enforce direct gravity summation for star-star gravity interactions *and* tree searches (e.g. for timestepping, binarity checks) within this radius *in AU*
#SINGLE_STAR_DIRECT_GRAVITY        # exact star-star (type 5) gravity at ALL separations, by brute-force summation over a table of every star broadcast to every task, with the tree dropping star mass for star targets so nothing is counted twice. Unlike the _RADIUS option above there is no cutoff, so the tree never approximates one star's pull on another. Costs O(N_star^2) per gravity step and stores O(N_star) per task, so it is for runs with few stars; incompatible with SINGLE_STAR_TIMESTEPPING>0 (whose binary search reads the star-star tree interactions this removes)
#ADAPTIVE_GRAVSOFT_MAX_SOFT_HARD_LIMIT=(1) # impose a hard upper limit (arbitrarily) to the softening kernel size for adaptive gravitational softening for gas cells
#ADAPTIVE_GRAVSOFT_SYMMETRIZE_FORCE_BY_AVERAGING # use the 'average the forces' implementation of force symmtrization in the gravity solver, instead of the default 'calculate forces for the larger softening', when two particles or cells with different force softening values interact inside each others kernels
# --------------------
# ----- Particle IDs
#TEST_FOR_IDUNIQUENESS          # explicitly check if particles have unique id numbers (only use for special behaviors)
#ASSIGN_NEW_IDS                 # assign IDs on startup instead of reading from ICs
#NO_CHILD_IDS_IN_ICS            # IC file does not have child IDs: do not read them (used for compatibility with snapshot restarts from old versions of the code)
# --------------------
# ----- Particle Merging/Splitting/Deletion/Boundaries
#PREVENT_PARTICLE_MERGE_SPLIT   # don't allow gas particle splitting/merging operations
#PARTICLE_EXCISION              # enable dynamical excision (remove particles within some radius)
#MERGESPLIT_HARDCODE_MAX_MASS=(1.0e-6)   # manually set maximum mass for particle merge-split operations (in code units): useful for snapshot restarts and other special circumstances
#MERGESPLIT_HARDCODE_MIN_MASS=(1.0e-7)   # manually set minimum mass for particle merge-split operations (in code units): useful for snapshot restarts and other special circumstances
#MHD_CONSERVE_B_ON_REFINEMENT      # redefine B after density step after a refinement/de-refinement operation so as to exactly conserve "B" between the steps, as compared to conserving the code "VB" between the steps [the default conserved integrated quantity]
# --------------------
# ----- Radiation-Hydrodynamics Special Options for Test Problems + Disabled or Other Special Features
#RT_DISABLE_UV_BACKGROUND               # disable extenal UV background in cooling functions (to isolate pure effects of local RT, or if simulating the background directly)
#RT_SEPARATELY_TRACK_LUMPOS             # keep luminosity vs. mass positions separate in tree. not compatible with Tree-PM mode, but it can be slightly more accurate and useful for debugging in tree-only mode with LEBRON or OTVET algorithms.
#RT_HYDROGEN_GAS_ONLY                   # sets hydrogen fraction to 1.0 (used for certain idealized chemistry calculations)
#RT_TIMESTEP_LIMIT_RECOMBINATION        # limit timesteps to the explicit recombination time when transporting ionizing photons. note our chemistry solvers are all implicit and can handle larger timesteps, but no gaurantee of transport accuracy for much larger steps since opacities depend on ionization states.
#RT_ENHANCED_NUMERICAL_DIFFUSION        # option which increases numerical diffusion, to get smoother solutions (akin to using HLL instead of HLLC+E fluxes), if desired; akin to slopelimiters~0 model
#RT_COMPGRAD_EDDINGTON_TENSOR           # forces computation of eddington tensor even when not needed by the code
#RT_REINJECT_ACCRETED_PHOTONS      # when sink particles are used, photons lost when a gas cell is accreted are reinjected into the lowest-energy frequency bin on the following photon injection from that sink
# --------------------
# ----- sink particle/black hole/star special options
#SINK_WIND_SPAWN_SET_BFIELD_POLTOR  # set poloridal and toroidal magnetic field for spawn particles (should work for all particle spawning). Cite Su et al., arXiv:2102.02206, for methods.
#SINK_WIND_SPAWN_SET_JET_PRECESSION # manually set precession in parameter file (does not work for cosmological simulations).  Cite Su et al., arXiv:2102.02206, for methods.
#SINK_RIAF_SUBEDDINGTON_MODEL       # enable an arbitrary modular variation in the radiative efficiency of BHs as a function of eddington ratio or other particle properties
#SINK_SCALE_SPAWNINGMASS_WITH_INITIALMASS # rescale the cell spawning mass criterion to scale with the initial sink mass for any sink and sink feedback model (instead of setting the spawning mass to a fixed universal constant in code units).
#SINK_SEED_FROM_LOCALGAS_TOTALMENCCRITERIA # modifies SINK_SEED_FROM_LOCALGAS to require that the local acceleration scale (including all mass/gravity sources) exceeds the critical value defined in arXiv:2103.10444. cite that paper for implementation
#SINK_INTERACT_ON_GAS_TIMESTEP      # perform the sink's gas interactions (neighbor search, accretion, feedback) only once about half the shortest neighboring-gas timestep has elapsed since the last ones, using the accumulated interval as their timestep
#SINK_GRAVCAPTURE_FIXEDSINKRADIUS   # uses a fixed sink radius for gravitational capture/accretion onto sink particles, taken to be the force softening kernel radius of the sink particle
#SINGLE_STAR_FB_TIMESTEPLIMIT     # limit the timesteps in gas cells potentially interacting with sink particles in the SINGLE_STAR modules, so that they cannot take timesteps large compared to quantities like the time for jets/winds to cross from the sink to the gas cell
# --------------------
# ----- Cosmic ray special options
#CRFLUID_ALT_SPECTRUM_SPECIALSNAPRESTART=1  # allows restart from a snapshot (flag=2) where single-bin CR model was used, for runs with CR spectra: the spectra are populated with the energy of the single-bin snapshot and fixed initial spectral shapes/ratios
#CRFLUID_ALT_DISABLE_LOSSES         # turn off CR heating/cooling interactions with gas (catastrophic losses, hadronic interactions, etc; only adiabatic/D:GradU work terms remain)
#CRFLUID_ALT_REACCEL_ONLY_DIFFUSIVE # replaces the correct form of the re-acceleration terms calculated from the focused CR transport equation with the more ad-hoc diffusive re-acceleration assumption that CR scattering/diffusion is dominated by an undamped, perfectly-symmetric, isotropic extrinsic turbulent cascade (not likely valid below ~TeV), following e.g. Drury+Strong 2017A&A...597A.117D; requires CRFLUID_EVOLVE_SPECTRUM.
#CRFLUID_ALT_VARIABLE_RSOL          # allows a variable (CR energy-dependent) reduced speed of light to be used for CRs, which is set in the function return_CRbin_M1speed defined by the user. cite Hopkins et al. 2021, arXiv:2103.10443
# --------------------
# ----- FIRE and STARFORGE sub-module special options
#FIRE_SNE_ENERGY_METAL_DEPENDENCE_EXPERIMENT # experiment with modified supernova energies as a function of metallicity - freely modify this module as desired. used for numerical experiments only.
#SINGLE_STAR_AND_SSP_HYBRID_MODEL=1 # cells with mass less than this (in solar) are treated with the single-stellar evolution models, larger mass with ssp models. needs user to specify a refinement criterion, and a criterion for when one module or another will be used. still in testing.
#SINGLE_STAR_AND_SSP_HYBRID_MODEL_DEFAULTS=1 # uses default settings for SINGLE_STAR_AND_SSP_HYBRID_MODEL=SINGLE_STAR_AND_SSP_HYBRID_MODEL_DEFAULTS (plus lots of other flag settings) from zoom-in experiments around special particles
#STARFORGE_GMC_TURBINIT             # special flag for custom ICs behavior in star formation simulations for turbulent clouds. adds an analytic uniform sphere harmonic potential + r^-3 halo outside to confine stirred turbulent gas, during the 'stirring' phase. cite Lane et al., 2022MNRAS.510.4767L
#STARFORGE_FILAMENT_TURBINIT        # special flag for custom ICs behavior in star formation simulations for turbulent clouds. adds an analytic potential of an finitite cylinder with a Plummer density profile, truncated at the ends of the cylinder, to confine stirred turbulent gas, during the 'stirring' phase. cite Lane et al., 2022MNRAS.510.4767L
#STARFORGE_FEEDBACK_TRACERS         # number of passive tracer fields for feedback channels (0=jets, 1=winds, 2=SNe); use 3
# --------------------
# ----- MPI & Parallel-FFTW De-Bugging
#DOUBLEPRECISION_FFTW           # FFTW in double precision to match libraries
#DISABLE_ALIGNED_ALLOC          # retired: the working memory pool is always aligned now. Blocks it hands out are rounded to a 32-byte boundary and some of them hold particle data, which requires that alignment, so the un-aligned variant was unsafe rather than merely slower. 'aligned_alloc' is standard in C11 and later, which this code requires anyway.
# --------------------
# ----- Load-Balancing
#ALLOW_IMBALANCED_GASPARTICLELOAD # retired and has no effect: the gas-cell capacity always follows the particle capacity, so the load-balancing freedom this used to enable is unconditional. A run with no gas allocates no gas-cell storage at all.
# --------------------
# ----- Development flags (in testing)
#GALSF_SFR_IMF_SAMPLING_DISTRIBUTE_SF=(2.0) #- star particle formation of O-stars is spread over this multiple of the free-fall time; requires GALSF_SFR_IMF_SAMPLING. developed by PFH
#GALSF_MERGER_STARCLUSTER_PARTICLES         #- module which merges star particles together meeting certain core-collapse conditions, so they can be cosmologically evolved. developed by PFH
#SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM           #- module for special nuclear zoom-in simulations. currently entirely custom behavior, not designed for wide use.
#SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM_TAG_ANCHOR #- derive the zoom refinement anchor (slot 0 of the SpecialParticle refinement array) from IC-tagged particles instead of a Type-3 special particle. reads a per-particle 'RefinementFlag' field from the ICs; particles with value 1 define the anchor, tracked as their mass-weighted center-of-mass (value=1, default) or the position of the single densest tagged gas cell (value=2). all downstream refinement/gravity/RT/sink consumers use the anchor unchanged. requires SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM. intended as a foundation for user-built refinement region logic.
#CRFLUID_INJECTION_AT_SHOCKS=(0.1)          #- inject CRs (using standard spectra) at resolved shocks throughout simulation. value=maximum fraction of shock energy flux to convert to accelerated CRs. currently using hard threshold of >Mach 5, >100 km/s shock velocity, along with spurious shock detection. coded by PFH, in testing
#SINK_CR_INJECTION_AT_TERMINATION=(0.25)    #- inject CRs (requires SINK_WIND_SPAWN and SINK_COSMIC_RAYS) approximately at spawned-cell termination shocks. developed by Kung-Yi Su, this version testing by PFH, currently uses very simple deceleration to fraction of launch velocity (=value set here) to determine when to inject
#SINK_TEST_WIND_MIXED_FASTSLOW=(1.e5)       #- BH has a slow outflow and fast jet, where fast jet has 1/100th the mass-loading and this speed in kms by default
#USE_TIMESTEP_DILATION_FOR_ZOOMS            #- enable time dilation modules, need to customize for applications, cannot be simply generically turned on without coding how they will work
#DILATION_FOR_STELLAR_KINEMATICS_ONLY       #- special version of time dilation designed for stellar kinematics in e.g. dense star clusters or galaxy centers
####################################################################################################

General De-Bugging and Special Behaviors

DEVELOPER_MODE: Turn this on to give yourself the option to set many numerical parameters explicitly in the run-time parameterfile. This includes things like the Courant factor and other error tolerance parameters. Otherwise these are hard-coded. Only enable this if you know what you are doing!!!

FORCE_EQUAL_TIMESTEPS: Force the code to take uniform timesteps (all elements in the same time-bin, set to the lowest value of any element in the timestep routine). This global timestepping is extremely inefficient for all but some very special problems (e.g. uniform sub-sonic turbulence), but very useful for de-bugging and testing conservation properties.

STOP_WHEN_BELOW_MINTIMESTEP: Forces code to quit when stepsize wants to go below MinSizeTimestep specified in the parameterfile.

CONTINUE_BELOW_MINTIMESTEP: Opts out of the above. STOP_WHEN_BELOW_MINTIMESTEP is on by default, because a run whose timestep has collapsed to the floor has almost always gone unstable, and allowing it to continue will burn an entire allocation making no progress. Setting this instead clamps the stepsize at MinSizeTimestep and carries on. Use it only for a problem where reaching the floor is both expected and survivable – if you are not certain that it is, you want the default.

Hydrodynamics

FREEZE_HYDRO: Zeros all fluxes from the Riemann problem, and don’t let particles move. This is a special flag useful for for testing additional physics layers in test problems where the normal hydro is not being tested.

EOS_ENFORCE_ADIABAT: If you turn this on, gas is forced to lie exactly along the adiabat specified. The numerical value is the value of the entropic function – i.e. at all timesteps, the internal energy is reset such that \(P=A\,\rho^{\gamma}\) where GAMMA_ENFORCE_ADIABAT=A sets the value of \(A\), \(P\) is pressure, and \(\rho\) is density, all in code units. Useful for test problems

HYDRO_REPLACE_RIEMANN_KT: This replaces the usual Riemann solver used for hydrodynamics (HLLC) with a ‘drop-in’ generalized Kurganov-Tadmor flux as derived in Panuelos, Wadsley, and Kevlahan in ‘Low Shear Diffusion Central Schemes for Particle Methods’, JCOMP, 2019. This is qualitatively similar to a Rusanov flux, as they are essentially Lax-Friedrichs schemes with some (substantial) improvements. As shown in their paper, the resulting fluxes are somewhat more diffusive than the default HLLC choice, but give smoother solutions (akin to changing the slope limiters) and as a result show better and more stable convergence properties on some problems. This module is compatible with either MFM, MFV, regular-grid, or any other hydro method except SPH (where enabling this will just do nothing), as it replaces the Riemann solver. It is currently not compatible with MHD, but it is straightforward to extend it to MHD if there is demand.

SLOPE_LIMITER_TOLERANCE: Enables finer control of the default slope-limiters used by the code. Default=1. 2 is the same as the ‘aggressive’ setting below. 0 is more conservative, which can be useful for problems (e.g. shearing boxes, subsonic turbulence) where maintaining good particle order and resolved gradients is more important the capturing very sharp phase discontinuities, for example

WAKEUP: Sets the timestep-limiter factor: a cell is ‘woken’ onto a shorter timestep, out of turn, when a neighbor’s timestep is shorter than its own by more than this factor. This is the limiter of Saitoh & Makino 2009, and it is what stops a cold cell sitting on a long timestep from being run over by a blastwave it never saw coming. Smaller values are more accurate and more expensive – their Table 1 gives a smaller energy error for f=2 than f=4 on the Sedov problem, at roughly twice the cost. If you do not set this explicitly the code follows SLOPE_LIMITER_TOLERANCE, taking 4.1 when that is positive and 2.1 otherwise.

ENERGY_ENTROPY_SWITCH_IS_ACTIVE: This enables the energy-entropy switch as described in GIZMO methods paper. This can greatly improve performance on some problems where the the flow is very cold and highly super-sonic. It can cause problems in multi-phase flows with strong cooling, though, and is not compatible with non-barytropic equations of state, so it is not enabled by default.

FORCE_ENTROPIC_EOS_BELOW: This allows you to set (manually) the alternative energy-entropy switch which is enabled by default in MFM/MFV: if relative velocities (approach mach numbers) are below this threshold relative to the sound speed, it uses the entropic EOS. This is designed to reduce numerical diffusivity much more accurately than the alternative ‘energy-entropy’ switch above, but this value can be important in highly sub-sonic problems

HYDRO_KERNEL_SURFACE_VOLCORR: Attempts to correct the kernel-estimated cell volumes (densities) for free-surface effects, for SPH, MFM or MFV. Near a free surface the kernel is only partly filled, so the density is under-estimated. The code measures the asymmetry of the neighbor distribution inside the kernel and divides the density by a correction factor between 0.34 and 1, using the boundary correction calibrated for the Wendland C2 kernel in Reinhardt & Stadel 2017 (arXiv:1701.08296; see their Fig. 3). It works with other kernels but is most accurate with that one. With this on, the face-closure error is used for this correction instead of for increasing the neighbor number of poorly-closed cells. It is enabled by default with EOS_ELASTIC (disable with DISABLE_SURFACE_VOLCORR).

DISABLE_SURFACE_VOLCORR: Disables HYDRO_KERNEL_SURFACE_VOLCORR if it would be set by default (e.g. if EOS_ELASTIC is enabled).

HYDRO_VOLUME_CORRECTIONS: Applies a higher-order partition-of-unity volume correction to the meshless cell volumes via a dedicated extra neighbor pass (the cellcorrections Spec). Unlike HYDRO_PARTITION_UNITY_IMPROVE_FD (which corrects only the leading \(\nabla H\) term, accumulated inline with the density loop), this pass evaluates a second-order quadrature correction over the full neighbor stencil. More accurate than the FD-only correction at the cost of an extra neighbor pass per (active) timestep; the two are complementary and can in principle be combined. Like HYDRO_PARTITION_UNITY_IMPROVE_FD, only meaningful for meshless (MFM/MFV) methods.

HYDRO_PARTITION_UNITY_IMPROVE_FD: Applies the first-derivative (FD) partition-of-unity volume correction from Massaro Acha, Alonso Asensio & Dalla Vecchia 2026, which corrects the meshless cell-volume estimates for the spatial gradient of the kernel support size \(H({\bf x})\). The default meshless volume partition \(V_{i} = 1 / \sum_{j} W(|{\bf x}_{i} - {\bf x}_{j}|, H_{i})\) is exact only in the limit of constant \(H\); the FD correction subtracts the leading \(\nabla H\) term, \(V_{i} \to V_{i} (1 + \alpha_{\zeta} {\bf \nabla} H \cdot \langle {\bf x}_{j} - {\bf x}_{i}\rangle / H + \ldots)\), with kernel-dependent coefficients \(\alpha_{\zeta}\) tabulated in their Table 1 (defined as \(\alpha_{\zeta} = -(2\pi/3) \int_{0}^{1} \xi^{5} (d\zeta/d\xi)\,d\xi\) where \(\zeta\) is the kernel function in their notation). This typically improves cubature-rule accuracy by a factor \(\sim 3\)–\(4\times\) at essentially zero extra cost, since the required gradient \(\nabla H\) is accumulated as part of the existing density neighbor loop (no separate neighbor pass). Only relevant for meshless (MFM/MFV) methods; ignored under SPH and regular-grid modes. Complementary to (but distinct from) HYDRO_VOLUME_CORRECTIONS, which uses a separate neighbor loop for a higher-order quadrature correction. Please cite Massaro Acha, Alonso Asensio & Dalla Vecchia 2026 if you use this module.

HYDRO_EXPLICITLY_INTEGRATE_VOLUME: Explicitly integrate the kernel continuity equation for cell volumes (giving e.g. densities), as in e.g. Monaghan 2000, but with a term that relaxes the integrated cell volume back to the explicitly evaluated kernel calculation on a timescale of \(\sim 10 t_{\rm cross}\) where \(t_{\rm cross} \sim {\rm MAX}(H_{\rm kernel} , L_{\rm grad}) / {\rm MIN}(cs_{\rm eff})\) where \(L_{\rm grad}\) is the density gradient scale length and \(cs_{\rm eff}\) the minimum sound/torsion/tension wave speed. This module ONLY makes sense for strictly fixed-mass (SPH/MFM) methods, do not enable it for others. It can be useful for some applications particularly of elastic materials, in weakly-compressible regimes.

DISABLE_EXPLICIT_VOLUME_INTEGRATION # disables HYDRO_EXPLICITLY_INTEGRATE_VOLUME if it would be set by default (e.g. if EOS_ELASTIC is enabled).

Additional Fluid Physics and Gravity

COOLING_OPERATOR_SPLIT: By default, if COOLING is active, the code will incorporate the hydro work terms into the cooling function which solves for the new temperature fully implicitly. However, enabling this explicitly operator-splits the hydro work update. This is less accurate, but necessary if e.g. you are enabling new cooling physics.

COOL_LOWTEMP_THIN_ONLY: In the COOL_LOW_TEMPERATURES module, treat low-temperature cooling as optically-thin instead of interpolating between optically-thin and -thick regimes.

MHD_ALTERNATIVE_LEAPFROG_SCHEME: Adopt an alternative leapfrog time-step scheme, where magnetic fields are treated like potential/positions (per Federico Stasyszyn’s suggestion): this is still being tested, but may give slightly improved results for certain types of problems (effects do appear to be small, however).

SUPER_TIMESTEP_DIFFUSION: Employs a super-timestepping scheme to accelerate integration of explicit diffusion operators. This can give a significant speedup on idealized problems. In fully non-linear problems, it is not clear how much this improves performance, and it needs to be carefully vetted for the given problem before it is used.

TURB_DRIVING_UPDATE_FORCE_ON_TURBUPDATE: Updates the turbulent driving force only as often as the driving phases themselves are recomputed, at the interval set by TurbDrive_TimeBetweenTurbUpdates, rather than every timestep. Enable this only as an optimization, when evaluating the turbulent force is itself expensive: to avoid large errors TurbDrive_TimeBetweenTurbUpdates must then be set by hand to be much smaller than \(\lambda_{\rm min}/V\), where \(V\) is the typical turbulent velocity and \(\lambda_{\rm min}\) the smallest driven wavelength.

EVALPOTENTIAL: Forces code to evaluate the gravitational potential (not just forces and accelerations) for all particles. Not needed unless de-bugging (even if you set the code to output the potential, you dont need to set this, since it will just evaluate when needed for outputs).

GRAVITY_HYBRID_OPENING_CRIT: Use both the Barnes-Hut and relative angle opening criterion for the gravity tree (normally choose one or the other). This is more conservative but allows more accurate forces and timestepping when local forces are strong enough to be dominant over intermediate-and-long-range forces, e.g. when there are hard binaries or multiples.

TIDAL_TIMESTEP_CRITERION: This replaces the standard acceleration-based timestep criterion with one based on the tidal tensor norm, which is more accurate and adaptive, when there are sharp passages with strong tidal forces (e.g. hard binaries). This only influences the gravity-based timestep, no other components of the timestepping. This of course also involves calculating the tidal tensor in the gravity tree. Currently this is designed only for Tree-based simulations: it neglects the PMGRID components, which are timestepped separately.

ADAPTIVE_TREEFORCE_UPDATE: This will use the tidal timescale to estimate how often gravity needs to be updated, updating a gas cell’s gravity no more often than ADAPTIVE_TREEFORCE_UPDATE (value set) times the tidal timestep otherwise used, with the default factor \(N_{f}\) in Grudic 2020 arxiv:2010.13792 (cite this if you are using these modules). Smaller is more accurate, larger is faster, should be tuned for your problem if used.

MAINTAIN_TREE_IN_REARRANGE: This will not rebuild the domains/tree every time a particle is spawned - salvage the existing one by redirecting pointers as needed. cite Grudic+ arXiv:2010.11254 for the technical methods. Useful if you are aggressively spawning on low-level timesteps.

RANDOMIZE_GRAVTREE: Move the top tree node around randomly so that treeforce errors are not correlated between one treebuild and another. cite Grudic+ arXiv:2010.11254

TREE_LEAF_BUCKET_SIZE: This sets the number of elements at which the code stops subdividing a tree node, so that a ‘leaf’ of the tree can hold several elements which the tree walk then evaluates directly, instead of splitting every branch all the way down to one element per leaf. Larger values give a smaller and shallower tree: cheaper to build, cheaper to store, cheaper to communicate between processors, and much less vulnerable to the pathologically deep branches that nearly-coincident elements would otherwise force. The trade-off is that more of the gravity and neighbor work is then done as direct element-element interactions, instead of being absorbed into the multipole moments of nodes. Note this does not reduce the force accuracy – if anything slightly the reverse, since a node the walk would have accepted as a multipole is now opened and summed exactly – the cost is purely in the number of interactions evaluated. Where the optimum lies is problem-dependent: strongly clustered, high-dynamic-range problems (deep trees, large numbers of nearly-empty nodes) favor larger values, while problems whose trees are already shallow gain little and can lose slightly. It is worth tuning if you care about performance. Very large values are limited in practice not by the extra interactions but by memory, since the portions of the tree exported to other processors grow as leaves are sent element-by-element. Setting this to 1 recovers exactly the historical behavior of one element per leaf.

TREE_QUERY_PACKET_SIZE: Sets how many adjacent active elements walk the gravity tree together as one ‘packet’. Members share a single traversal, but each still makes its own node-opening decisions and evaluates its own accepted nodes and particles, so the forces are unchanged. What is shared is the cost of descending a deep tree. Default 8; set to 1 to walk every element alone. Values up to 256 are taken as one packet, and larger values are walked as several packets. The best value is problem-dependent: larger packets help most on steps with deep trees, but cost more opening-criterion work on the many steps with few active elements. On the GPU the team of threads walking a packet is generally wider than the packet, so this sets how many elements share a traversal, not how many threads perform it. The count is also capped by the team size. A device call whose elements reach about half the device’s lanes walks one element per lane instead, so on the device this matters only below that; the host walk is unaffected. The packet shape actually used is reported in the per-call record in timings.txt (packet: Q= T= Qdev= walkers= ...).

GRAVITY_SPHERICAL_SYMMETRY: Modifies the tree gravity solver to give the solution assuming spherical symmetry about the origin (if BOX_PERIODIC is not enabled) or the box center - this works by simply summing all the mass in a sphere from the current position to the defined box center, and then taking the acceleration for a spherically symmetric distribution. Useful for IC generation and test problems. Numerical value specifies a minimum softening length. (cite Lane et al., arXiv:2110.14816)

SINGLE_STAR_DIRECT_GRAVITY_RADIUS: For sink (type 5) targets, forces the gravity tree to open any node containing sinks whose center lies within this distance (in AU, plus the usual node-size margin). Star-star forces inside this radius, and the tree searches that use them (e.g. nearest-star distances for timestepping and binary identification), are then computed by direct summation over individual sinks rather than from node multipoles. Active when SINGLE_STAR_TIMESTEPPING or SINGLE_STAR_FIND_BINARIES is defined. Default 1000 AU under SINGLE_STAR_STARFORGE_DEFAULTS; set a different value to change it. Unlike SINGLE_STAR_DIRECT_GRAVITY below, interactions beyond this radius are still taken from the tree.

SINGLE_STAR_DIRECT_GRAVITY: Computes star-star (type 5) gravity exactly at -all- separations, by brute-force summation over a table of every star broadcast to every task, with the tree dropping star mass for star targets so that nothing is counted twice. Unlike the SINGLE_STAR_DIRECT_GRAVITY_RADIUS option, there is no cutoff, so the tree never approximates one star’s pull on another at any distance. This costs \(O(N_{\rm star}^{2})\) per gravity step and stores \(O(N_{\rm star})\) per task, so it is intended for runs with a modest number of stars. It is incompatible with SINGLE_STAR_TIMESTEPPING>0, whose binary search reads exactly the star-star tree interactions that this removes.

ADAPTIVE_GRAVSOFT_MAX_SOFT_HARD_LIMIT: With adaptive gravitational softening for gas (ADAPTIVE_GRAVSOFT_FORGAS, or the timestep softening used with SELFGRAVITY_OFF), caps the gas-cell softening at this value, in physical code length units. Normally the softening is set equal to the hydro kernel radius. The cap is arbitrary: it simply prevents very large softenings in very low-density gas. It is set to 0.1 kpc by default with FIRE_PHYSICS_DEFAULTS=3.

ADAPTIVE_GRAVSOFT_SYMMETRIZE_FORCE_BY_AVERAGING: Changes how the gravitational force is symmetrized when two elements with different softenings interact inside each other’s kernels. The default evaluates the pair force with the larger of the two softenings. With this on, the force is the average of the forces evaluated with each element’s own softening. This applies only to pairs whose types both use adaptive softening (ADAPTIVE_GRAVSOFT_FORALL bitmask); other pairs still use the larger softening, as do pairs involving sinks under SINGLE_STAR_SINK_DYNAMICS. The same choice is used in the ‘zeta’ correction terms for adaptive softening, which assume averaging. For this reason it is enabled automatically with ADAPTIVE_GRAVSOFT_FORALL.

Particle IDs

TEST_FOR_IDUNIQUENESS: Forces the code to check at startup that all particles have unique IDs.

ASSIGN_NEW_IDS: Make the code assign particles new IDs on startup. This is useful if the ICs do not contain IDs for some reason, or they if one needs to use longer IDs. They will not be assigned in any particular order (just the initial particle ordering from the IC read).

NO_CHILD_IDS_IN_ICS: Enable this if you are restarting from a snapshot or restart file and the IC file does not have child IDs: this will force the code not to attempt to read them (used for compatibility with snapshot restarts from old versions of the code)

Particle Merging/Splitting/Deletion/Boundaries

PREVENT_PARTICLE_MERGE_SPLIT: Do not allow particles to merge or split over the course of the simulation. This is generally useful only for MFV runs, where one wishes to prevent these operations in order to de-bug (check whether splitting/merging is causing problems for the runs).

PARTICLE_EXCISION: Enable dynamical particle excision (removal of particles within/outside of some radius). The user must manually hard-code the conditions for excision. Do so by searching the source code for this flag, and you will find the relevant function where the excision occurs.

MERGESPLIT_HARDCODE_MAX_MASS/MERGESPLIT_HARDCODE_MIN_MASS: Manually set the maximum/minimum allowed mass for particle merge-split operations, to the value set (in code units). This is useful for snapshot restarts and other special circumstances, when the automated routine (which simply reads the input/snapshot, takes the most/least massive particle and multiples by a safety factor of ~3 in either direction) may produce a range of particle masses larger than desired.

MHD_CONSERVE_B_ON_REFINEMENT: After a refinement or de-refinement operation, redefines \({\bf B}\) following the density step so that \({\bf B}\) itself is exactly conserved across the operation. The default instead conserves the code’s volume-integrated quantity \({\bf VB}\) between the steps, which is the natural conserved variable for the transport but does not hold the field strength itself fixed when the cell volume changes.

PARTICLE_MERGE_SPLIT_EVERY_TIMESTEP: Force merge/split operations to occur every timestep, instead of only on domain decomposition steps

Radiation-Hydrodynamics Special Options for Test Problems

RT_DISABLE_UV_BACKGROUND: Disable extenal UV background in cooling functions (to isolate pure effects of local RT, or if simulating the background directly).

RT_SEPARATELY_TRACK_LUMPOS: Keep luminosity vs. mass positions separate in tree. not compatible with Tree-PM mode, but it can be slightly more accurate and useful for debugging in tree-only mode with LEBRON or OTVET algorithms.

RT_HYDROGEN_GAS_ONLY: Sets hydrogen fraction to 1.0 (used for certain idealized chemistry calculations).

RT_TIMESTEP_LIMIT_RECOMBINATION: Limit timesteps to the explicit recombination time when transporting ionizing photons. note our chemistry solvers are all implicit and can handle larger timesteps, but no gaurantee of transport accuracy for much larger steps since opacities depend on ionization states.

RT_ENHANCED_NUMERICAL_DIFFUSION: Option which increases numerical diffusion, to get smoother solutions (akin to using HLL instead of HLLC+E fluxes), if desired. This is akin to the slopelimiters=0 model, for the radiation.

RT_COMPGRAD_EDDINGTON_TENSOR: Forces computation of eddington tensor even when not needed by the code.

RT_REINJECT_ACCRETED_PHOTONS: When sink particles are used, photons which would otherwise be lost with a gas cell when it is accreted are instead re-injected into the lowest-energy frequency bin at the next photon injection from that sink. Without this the radiation carried by accreted cells simply disappears, which is a small but systematic photon sink in runs where accretion is rapid.

Black Hole/Sink Particle Special Options

SINK_WIND_SPAWN_SET_BFIELD_POLTOR: Set the poloridal and toroidal magnetic field for spawned jet cells (should work for all cell spawning). Cite Su et al., arXiv:2102.02206, for methods.

SINK_WIND_SPAWN_SET_JET_PRECESSION: Manually set jet precession in parameter file (does not work for cosmological simulations). Will make the jet direction precess according to a user-specified precession angle and period. Cite Su et al., arXiv:2102.02206, for methods.

SINK_RIAF_SUBEDDINGTON_MODEL: Enable an arbitrary modular variation in the radiative efficiency of BHs as a function of eddington ratio or other particle properties. This is determined by the function evaluate_sink_radiative_efficiency in sink_util.c, which the user should set to scale according to the desired model.

SINK_SCALE_SPAWNINGMASS_WITH_INITIALMASS: With SINK_WIND_SPAWN, makes the parameter Sink_outflow_particlemass (the mass of each spawned cell) a fraction of the sink’s formation mass instead of a fixed mass in code units. If the parameter is not set, it defaults to 0.01. This is purely a convention choice. It is mainly useful in multi-resolution (hyper-refinement) runs, where sinks form at very different masses and a single fixed spawning mass is inappropriate. It is enabled by default for FIRE_BHS with FIRE_PHYSICS_DEFAULTS > 2.

SINK_SEED_FROM_LOCALGAS_TOTALMENCCRITERIA: Modifies the SINK_SEED_FROM_LOCALGAS seeding probability. The default uses the Sobolev-estimated column density of gas alone. With this flag the surface density is computed in the gravity tree walk from the total enclosed mass (all mass and gravity sources) within \(R_{\rm crit} = {\rm MAX}(\epsilon,\ 0.1\,{\rm kpc})\) of the cell, where \(\epsilon\) is its force softening. The seed probability then falls off quadratically below a total surface density of 1 g cm${–2}$, which sets a minimum local acceleration scale. It also falls off as \(Z^{-2}\) above 0.1 solar metallicity. If SINK_CALC_DISTANCES is on (without SPECIAL_POINT_MOTION), no seed forms within \(10\,R_{\rm crit}\) of an existing sink. Enabled by default with FIRE_BHS. Cite arXiv:2103.10444 for the implementation.

SINK_INTERACT_ON_GAS_TIMESTEP: Ties the frequency of sink-gas interactions (neighbor search, accretion, and feedback injection) to the timestep of the surrounding gas rather than of the sink itself. A sink performs these operations only once the time since its last gas interaction exceeds about half the timestep of its shortest-timestep gas neighbor (or always, for a star going supernova under the STARFORGE modules). The accumulated interval is then used as the timestep for accretion and feedback. Sinks that must take very short steps for their own N-body dynamics therefore do not repeat the more expensive gas operations every step. Enabled by default with SINGLE_STAR_SINK_DYNAMICS.

SINK_GRAVCAPTURE_FIXEDSINKRADIUS: Changes the gravitational-capture accretion criterion to a Bate et al. 1995-style fixed sink radius. Each sink carries its own radius, distinct from both its force softening and its neighbor-search radius. Gas that is gravitationally bound to the sink and inside this radius is accreted (grains inside 1.4 times this radius). The radius is set when the sink forms, to the force-softening kernel radius of the parent cell. For single-star sinks it is the larger of that and \(0.79\,G M/c_{s}^{2}\), with \(c_{s}=0.2\) km/s (scaled as \(n^{1/5}\) above \(n_{\rm H}=10^{10}\,{\rm cm^{-3}}\) where the gas becomes opacity-limited). It is then held fixed; in cosmological runs it is capped at the sink’s gravitational kernel radius. The radius is saved to snapshots (HDF5 “SinkRadius”) and can be read back with INPUT_READ_SINKPROPS. It also sets the minimum launch distance for spawned cells and enters the sink timestep criteria. Enabled automatically by SINGLE_STAR_ACCRETION=12 (the STARFORGE default). Cite Grudic et al., arXiv:2010.11254.

SINGLE_STAR_FB_TIMESTEPLIMIT: In the SINGLE_STAR modules, limits the timestep of gas cells that may interact with feedback from sinks. Each sink carries its maximum feedback velocity (jets, winds, or SN ejecta), and the gravity tree records, for every gas cell, the minimum over sinks of (softened distance)/(that velocity). The cell’s timestep is limited to 0.5 times the Courant factor times the smaller of this feedback crossing time and the sink approach time. The cell therefore cannot take a step longer than it takes ejecta to reach it from the sink. The same feedback velocity is added to the signal speed in the sink’s own Courant timestep. This requires SINGLE_STAR_TIMESTEPPING. It is enabled automatically when any single-star feedback (or explicit RT) module is on together with SINGLE_STAR_TIMESTEPPING and SINGLE_STAR_SINK_DYNAMICS. Cite Grudic et al., arXiv:2010.11254.

SINK_CR_SUBEDDINGTON_MODEL: Enable an arbitrary modular variation in the CR acceleration efficiency from BHs as a function of mass, eddington ratio, spin, or other particle properties. This is determined by the function evaluate_sink_cosmicray_efficiency in sink_util.c, which the user should set to scale according to the desired model.

Cosmic Ray Special Options

CRFLUID_ALT_SPECTRUM_SPECIALSNAPRESTART: Allows restart from a snapshot (flag=2) where single-bin CR model was used, for runs with CR spectra: the spectra are populated with the energy of the single-bin snapshot and fixed initial spectral shapes/ratios.

CRFLUID_ALT_DISABLE_LOSSES: Turn off CR heating/cooling interactions with gas (catastrophic losses, hadronic interactions, etc; only adiabatic or D:GradU work terms remain). Obviously this is not physical but is sometimes useful for testing.

CRFLUID_ALT_REACCEL_ONLY_DIFFUSIVE: Adds the ad-hoc ‘diffusive re-acceleration’ term to the CR spectral evolution. This assumes CR scattering is dominated by an undamped, perfectly symmetric, isotropic extrinsic turbulent cascade, following e.g. Drury & Strong 2017, A&A, 597, A117. The momentum-space diffusion coefficient is set from the spatial diffusion coefficient via the usual heuristic \(D_{xx} D_{pp} \sim v_{A}^{2}\). The local logarithmic slope of \(\kappa\) with rigidity is estimated from neighboring spectral bins. The term is applied (and limits the spectral sub-cycling timestep) only once \(\kappa > 10^{25}\,{\rm cm^{2}\,s^{-1}}\). This form is not likely valid below ~TeV energies. It is the conventional assumption rather than the re-acceleration that follows from the focused CR transport equation, so it is provided for comparison with that literature. Requires CRFLUID_EVOLVE_SPECTRUM.

CRFLUID_ALT_VARIABLE_RSOL: Turning this on allows the user to define a variable (CR rigidity-dependent, so requires CRFLUID_EVOLVE_SPECTRUM) reduced speed of light to be used for CRs, which is set in the function return_CRbin_M1speed defined by the user in cosmic_ray_utilities. cite Hopkins et al. 2021, arXiv:2103.10443 for the numerical methods and analytic derivation of the method showing that this can be done. Caution is needed with this, to ensure the RSOL still obeys all the relevant constraints, and also, importantly, that it does not lead to some bias in spectral shapes or numerical instability in certain regimes, which can occur if the RSOL varies too rapidly. Users should test this before it is used too heavily.

FIRE Module Special Options

FIRE_SNE_ENERGY_METAL_DEPENDENCE_EXPERIMENT: Experiment with modified supernova energies as a function of metallicity - freely modify this module as desired. used for numerical experiments only.

SINGLE_STAR_AND_SSP_HYBRID_MODEL: Cells with mass less than this value (in solar) are treated with the single-stellar evolution models from STARFORGE, larger mass with ssp models from FIRE. needs user to specify a refinement criterion, and a criterion for when one module or another will be used. still in testing. contact the developers of both module sets before using it.

SINGLE_STAR_AND_SSP_HYBRID_MODEL_DEFAULTS: Sets a bundle of defaults for the hybrid model above, taken from the zoom-in experiments around special particles, so that such a run can be configured from a single flag; the value is passed through as SINGLE_STAR_AND_SSP_HYBRID_MODEL itself, i.e. the mass resolution below which the single-star routines are used. Beyond that it enables SINGLE_STAR_STARFORGE_DEFAULTS and GALSF_SFR_IMF_SAMPLING (so the intermediate-mass regime is at least approximately handled), turns on COOLING, MAGNETIC, CONDUCTION and CONDUCTION_SPITZER, VISCOSITY and VISCOSITY_BRAGINSKII, enables protostellar jets, continuous winds, SNe and radiative feedback (SINGLE_STAR_FB_JETS, SINGLE_STAR_FB_WINDS, SINGLE_STAR_FB_SNE, SINGLE_STAR_FB_RAD), and configures the transport with RT_COMOVING and RT_SOURCES=(16+32) so that both single-star and stellar-population particles emit, a reduced speed of light of 0.1 unless one is set explicitly, and ADAPTIVE_TREEFORCE_UPDATE=0.0625. Set the individual flags yourself if you want any combination other than this one.

STARFORGE_GMC_TURBINIT: Special IC-generation behavior for star formation simulations of turbulent clouds. During the turbulent ‘stirring’ phase used to generate the ICs, it adds an analytic confining potential to keep the stirred gas together: a uniform sphere of mass \(0.808\,M_{\rm gas,total}\) and radius \(L_{\rm box}/10\) centered in the box (the defaults of the MakeCloud uniform-sphere ICs; edit the code for other setups), plus an \(r^{-3}\) density halo outside it. The value multiplies the strength of the potential and should be set to the desired virial parameter. With the STARFORGE defaults this automatically enables TURB_DRIVING, GRAVITY_ANALYTIC and SELFGRAVITY_OFF. Cite Lane et al., 2022, MNRAS, 510, 4767.

STARFORGE_FILAMENT_TURBINIT: As STARFORGE_GMC_TURBINIT, but the confining potential during the stirring phase is that of a finite cylinder along the \(x\)-axis with a Plummer-like (\(p=3\)) radial density profile, truncated at the ends of the cylinder (using an approximate expression for the axial field). The value again multiplies the potential strength (the desired virial parameter). The filament mass, radius and length are taken from STARFORGE_FILAMENT_MASS, STARFORGE_FILAMENT_RADIUS and STARFORGE_FILAMENT_LENGTH (code units), which are not defined anywhere in the code, so you must define them in Config.sh yourself. Same automatic flags as above. Cite Lane et al., 2022, MNRAS, 510, 4767.

STARFORGE_FEEDBACK_TRACERS: Adds passive tracer fields in the SINGLE_STAR modules that track how much of each cell’s gas came from each feedback channel: field 0 for protostellar jets, 1 for main-sequence winds, 2 for supernovae. Ejecta from each channel are spawned with that channel’s tracer equal to 1 and the others 0; the tracers are then mixed like metals. They are stored in the metallicity array after the standard species, R-process and age tracers, and are labelled with key \(-3\) in the snapshot attribute “Metals_Atomic_Number_Or_Key”. The value is the number of tracer fields allocated. The code writes all three channels, so use 3 (the value set automatically with the STARFORGE defaults whenever jets, winds or SNe are on, which also turns on TURB_DIFF_METALS and TURB_DIFF_METALS_LOWORDER).

MPI & Parallel-FFTW De-Bugging

DOUBLEPRECISION_FFTW: If this is set, the code will use the double-precision version of FTTW, provided the latter has been explicitly installed with a “d” prefix, and NOTYPEPREFIX FFTW is not set. Otherwise the single precision version (“s” prefix) is used.

DISABLE_ALIGNED_ALLOC: Disable calls to aligned_alloc, needed for older C99-only versions of GCC compilers (everything C11+ -should- be compatible and not need this)

Load-Balancing

ALLOW_IMBALANCED_GASPARTICLELOAD: Increases All.MaxPartSph (maximum number of gas cells allowed per MPI task, default is PartAllocFac*Number of gas particles/Number of Tasks) to All.MaxPart (same using all particles instead of gas elements): can allow better load-balancing in some cases, but uses more memory. But try this flag if you run into errors where it can’t fit the domain (where you would increase PartAllocFac, but can’t for some reason). It can help to declare ‘up front’ the memory useage, preventing this error even though the mean memory footprint is larger. This is essentially required if you are running simulations that involve lots of element-splitting, as one is continuously creating new gas resolution elements, which will overflow memory because of how they have to be ranked.

SEPARATE_STELLARDOMAINDECOMP: This will completely separate the star particles in the domain decomposition and memory structures. The implementation is incomplete – it should only be used for specific debugging.

Development Flags (In Testing)

These modules are in development and have not been validated for general use; there is no methods paper for them yet. Contact the developer named in each entry (or the code developers, if none is named) before using any of them.

GALSF_SFR_IMF_SAMPLING_DISTRIBUTE_SF: Modifies GALSF_SFR_IMF_SAMPLING (which it requires). Normally a new star particle’s number of massive (O) stars is drawn all at once when it forms. With this flag a new particle starts with none, and the massive stars are drawn stochastically (Poisson, at the same mean of ~1 per 100 \(M_{\odot}\)) on each of its timesteps. This continues over a period equal to this value times \(0.54/\sqrt{G\rho}\) of the parent gas (roughly that many free-fall times), approximating star formation spread out in time within the particle. The particle’s effective stellar age used for feedback is the number-weighted mean formation time of its massive stars. Its luminosity is scaled by the sampled massive-star number. The spreading time is saved to snapshots (HDF5 “TimeDistribOfStarFormation”). Developed by PFH; in testing.

GALSF_MERGER_STARCLUSTER_PARTICLES: Allows star particles (type 4) to merge with one another when they meet ‘core-collapse’-like conditions, so that dense star clusters can be represented by single particles and evolved cosmologically without following their internal dynamics. A star particle becomes eligible only once it is older than 50 Myr (after the end of any GALSF_SFR_IMF_SAMPLING_DISTRIBUTE_SF formation period) and below \(10^{6}\,M_{\odot}\). It must also sit in a very dense region, judged from the tidal tensor if that is computed, or otherwise from a timestep below 0.01 Myr. It then merges with the nearest star particle that is within the larger of the two softenings and bound to it. Each star particle carries an effective size, initialized at formation to the smaller of the parent gas softening and the type–4 softening and updated on mergers from the pair’s total gravitational energy. This size is used as its gravitational softening. Developed by PFH; in testing.

SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM: Module for special nuclear zoom-in (re-simulation) runs centered on one or more ‘special’ type–3 particles. These particles define the refinement centers used by the refinement, gravity-boundary, RT and sink routines. The value is the number of such centers (default 1). The behavior is entirely custom and not designed for wide use. Among other things, the type–3 particles are kept on the smallest occupied timestep (with a maximum of 1000 yr) and their accreted mass is tracked. Collisionless particles are not allowed to merge. Type–4 softenings scale with particle mass. With SINGLE_STAR_AND_SSP_HYBRID_MODEL_DEFAULTS it also enables PARTICLE_EXCISION and a set of additional outputs. Combined with USE_TIMESTEP_DILATION_FOR_ZOOMS, it supplies the dilation profile around the centers. Incompatible with RANDOMIZE_GRAVTREE under periodic gravity. Contact the code developers before using it.

SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM_TAG_ANCHOR: With SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM (required), takes the first refinement center (anchor) from particles tagged in the ICs instead of from a type–3 special particle. The ICs must contain a per-particle integer field “RefinementFlag”; particles with value 1 are tagged, and particles in files without the field are untagged. Every timestep the anchor is recomputed as the mass-weighted center of mass of all tagged particles (value 1, the default) or as the position of the single densest tagged gas cell (value 2). Everything downstream that uses the refinement center uses this anchor unchanged. The tag is carried through and written to snapshots. It is intended as a foundation for user-built refinement-region logic. In testing; contact the code developers before use.

CRFLUID_INJECTION_AT_SHOCKS: Injects CRs (with the standard injection spectra) at shocks resolved anywhere in the simulation; requires COSMIC_RAY_FLUID. Shocks are identified pairwise at hydro faces. The flow must be converging at more than 100 km/s, the internal energy, density and pressure jumps must all have the same sign (to reject spurious shocks), and the jumps must imply a Mach number above 5 from the temperature jump and above 1.3 from the pressure jump. The injected CR power is the value set times an estimate of the dissipated shock energy flux (from the upwind density and face velocity). With MAGNETIC it is also multiplied by an obliquity factor that suppresses injection at quasi-perpendicular shocks. At most half of each cell’s net thermal-energy gain in the step is moved into CRs, and that amount is removed from the gas so total energy is conserved. The value is thus the maximum fraction of the shock energy flux converted to CRs. Coded by PFH; in testing.

SINK_CR_INJECTION_AT_TERMINATION: With SINK_WIND_SPAWN and SINK_COSMIC_RAYS, the CR energy assigned to each spawned cell is not injected at launch but stored on the cell. It is injected (directed down the local pressure gradient) once the cell has decelerated below this value times the launch velocity, approximating injection at the outflow’s termination shock. The launch velocity is Sink_outflow_velocity, or the jet speed under SINK_TEST_WIND_MIXED_FASTSLOW or SINK_RIAF_SUBEDDINGTON_MODEL. The stored energy is shared appropriately if the cell is split. This deceleration criterion is deliberately very simple. Developed by Kung-Yi Su; this version is in testing by PFH.

SINK_TEST_WIND_MIXED_FASTSLOW: Test module for SINK_WIND_SPAWN from BH/sink (type 5) particles, which launches a mixed slow outflow and fast jet. Spawned cells are launched in pairs. For each pair there is a 10% chance it is a collimated jet pair, with cells of 1/100 the mass of the other cells, launched at this value (in km/s; default \(10^{5}\)). Otherwise the pair is the usual broad-angle wind at the standard launch velocity. Cell masses are normalized so the total spawned mass is unchanged. With SINK_COSMIC_RAYS, CR energy is carried only by the jet cells, boosted by the mass ratio to keep the total. In testing.

USE_TIMESTEP_DILATION_FOR_ZOOMS: Enables the timestep-dilation framework for zoom-in simulations. Each particle (and tree node) carries a dilation factor \(f = 1/a \le 1\), frozen when its step is assigned. One unit of integer time then corresponds to \(f\) times the normal physical time for that particle, so regions far from the zoom target are integrated in slow motion and take correspondingly longer integer steps. The dilation profile \(a\) is not generic: it must be coded for the application in timestep_dilation_factor (core/timestep.cc). As provided, \(a=1\) (no effect) unless SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM is set, which gives \(a = 1 + 1/(1/100 + r/r_{\max})\) with \(r\) the distance to the nearest refinement center and \(r_{\max} = 100\) times the type–3 softening. Otherwise \(a\) comes from a smoothing weight under SPECIAL_POINT_WEIGHTED_MOTION. It disables the default HERMITE_INTEGRATION of the STARFORGE settings. It cannot simply be switched on generically. In testing; contact the code developers before use.

DILATION_FOR_STELLAR_KINEMATICS_ONLY: A variant of USE_TIMESTEP_DILATION_FOR_ZOOMS (which it turns on) for stellar kinematics in e.g. dense star clusters or galaxy centers. Only star particles (type 4, plus the special smoothing-source type under SPECIAL_POINT_WEIGHTED_MOTION) are dilated. Gas, other types, tree nodes and feedback timesteps are not. For a dilated star, the drift and kick over the undilated remainder of each interval add back the velocity and acceleration of the nearest special particle. Only the star’s motion relative to its surroundings is slowed; its bulk motion with them is not. As with the parent flag, the dilation profile must be set up for the application. In testing; contact the code developers before use.

8. The Parameterfile (Setting run-time parameters)

The other thing required to get your run going is a parameterfile in the run directory - this is the file we called “myparameterfile.params” in our guide to using the code. This contains all the run-time parameters for the code. Basically, anything not set in the compile-time flags (in Config.sh) needs to be set in this file. When the run starts, the file will be parsed, and these values will be used for the run. Many of them can be changed on restarts (all can be changed on restarts from snapshots), but you normally don’t need to make any changes to the parameterfile when restarting.

In this file, each parameter value is set by specifying a keyword, followed by a numerical value or a character string, separated by whitespace (spaces, tabs) from the keyword. For each keyword, a separate line needs to be used, with the value and keyword appearing on the same line. Between keywords, arbitrary amounts of whitespace (including empty lines) may be used. The sequence of keywords is arbitrary, but each keywords needs to appear exactly once, even if its value is not relevant for the particular simulation (otherwise you’ll get an error message). Note that the keywords are type-sensitive. Lines with a leading ‘%’ are ignored. In lines with keywords, comments may also be added after the specification of the value for the corresponding keyword. Here we will show (piece-by-piece) an example parameterfile, and then discuss each keyword and the meaning of its value.

The example parameterfile is included with the source code, in the folder “scripts” as “params.txt”

Generic (Always-Required) Parameters

GIZMO will only require parameters be set in the parameterfile if they are required by the different physics modules enabled in the compiled version of the code (e.g. your compile-time flags). If a parameter in your parameterfile is not required, the code will simply ignore it – but it will spit out a warning telling you it is ignoring the parameter, so you can be sure this is intended (e.g. if its telling you it is ignoring the parameters for radiation-hydro, and you intended those flags to be on, then you know something has gone wrong with your compilation).

In this section, we discuss all the parameters that are always ‘active’ potentially, regardless of extra physics (we discuss parameters required by additional physics moduled below). However, some of these parameters are only required if certain other flags are set, or only required for certain non-standard behaviors, etc. In many cases the code will try to guess sensible default values for these parameters if you do not specify them. Again, it will warn you it is doing so in the lines of the standard output immediately following parsing the parameter file, and tell you what the default values it is assuming are. Be sure to check this, to be absolutely sure you are running with the parameter values you think you are using!

For convenience and quality-of-life reasons, as well as historical compatibility, some of these parameters will accept any of multiple names in the parameterfile (e.g. there might be a name which is more descriptive and useful based on GIZMO conventions and this user guide, but you can also often use the old GADGET–2 name for a parameter if you want, for parameters that were defined in those older codes). You just choose one to use based on your preference.

As a result of this, a ‘minimal’ parameterfile can be as short as the following:

InitCondFile                nameofIC    % name of the ICs file  
ICFormat                    3           % format of ICs (hdf5)  
OutputDir                   output      % name of output directory  
PartAllocFactor             2           % max particle-load imbalance  
ComovingIntegrationOn       0           % is it cosmological?  
BoxSize                     1           % box size in code units  
TimeBegin                   0           % initial simulation time  
TimeMax                     1           % final simulation time  
OutputListOn                0           % use tabulated snap list?  
TimeOfFirstSnapshot         0           % time for first snapshot  
TimeBetSnapshot             0.1         % time between snapshots  
MaxSizeTimestep             0.01        % maximum allowed timestep  
DomainBuild_ActiveFraction  0.01        % fraction active for decomposition  
TreeRebuild_ActiveFraction  0.01        % fraction active for tree build  
DesNumNgb                   32          % effective neighbor number  
Softening_Type0             0.01        % gas (type=0) force-softening  
Softening_Type1             0           % type=1 force-softening  
Softening_Type2             0           % type=2 force-softening  
Softening_Type3             0           % type=3 force-softening  
Softening_Type4             0           % type=4 force-softening  
Softening_Type5             0           % type=5 force-softening

This example sets every strictly-necessary parameter if we are running a non-cosmological, pure-hydrodynamics simulation (e.g. simulating a Kelvin-Helmholtz instability or a number of other example test problems included in this User Guide). Of course, things like nameofIC need to be set to the correct values: we explain what all of these actually are below, as well as many additional parameters not set here needed for other behaviors. But this gives you all the basics. You see each parameter name (e.g. InitCondFile or ICFormat), value set (e.g. nameofIC or 3), and some short comments (preceded by the % character) describing them. Some flags tell the code about directory structures and where to find and write things (and when to do so), others tell it about the machine on which we are running and allow us to optimize load-balancing and memory, others specify physics which is or is not active, others tell it purely-numerical constraints like force-softening and kernel neighbor numbers.

Filenames, Directories, & Formats

%----  Relevant files (filenames and directories)  
InitCondFile  /pathname/to/my/ic/file/filename_without_hd5_extension  
OutputDir     output

InitCondFile (or Initial_Conditions_File): This sets the filename of the initial conditions to be read in at start-up. Note that the initial conditions file (or files) does not have to reside in the output directory. The initial conditions can be distributed into several files, in the same way as snapshot files. In this case, only the basename without the tailing .n number should be specified as initial conditions filename. The code will recognize the number of files that make up the initial conditions from the header entries, and load all of these files accordingly. There is no constraint for the number of these files in relation to the processor number.

OutputDir (or Output_Directory): This is the pathname of the directory that holds all the output generated by the simulation (snapshot files, restart files, diagnostic files). Note that this directory needs to exist, otherwise an error message will be produced.

%---- File formats (input and output)  
ICFormat    3  % 1=unformatted (gadget) binary, 3=hdf5, 4=cluster  
SnapFormat  3  % 1=unformatted (gadget) binary, 3=hdf5

ICFormat (or Initial_Conditions_Format): This flag selects the file format of the initial conditions read in by the code upon start-up. A value of 1 selects the standard unformatted binary file-format of GADGET–2 (which is identical to the unformatted binary format used for GIZMO), while a value of 2 or 4 selects a more uncommonly-used variant of this simple binary format (also identical between GADGET/GIZMO). A value of 3 selects the use of HDF5 binary instead (again, same for GADGET/GIZMO). The structure of these files will be discussed in a separate section below. This should always match the format of the file named in “InitCondFile”. If not specified, this will default to type 3 (hdf5), but many codes widely-used to generate ICs will generate format = 1 ICs, so you may need to set this in many of your applications.

SnapFormat (or Snapshot_Format): A flag that specifies the file-format to be used for writing snapshot files. The possible format choices are the same as for ICFormat. It is therefore possible to use different formats for the initial conditions and the produced snapshot files. If not specified, this will default to type 3 (hdf5).

Output Parameters

%---- Output parameters  
RestartFile               restart  
SnapshotFileBase          snapshot  
OutputListOn              1 % =1 to use list in "OutputListFilename"  
OutputListFilename        output_times.txt  % list of times (in code units) for snaps  
NumFilesPerSnapshot       1  
NumFilesWrittenInParallel   16  % must be < N_processors & power of 2

RestartFile (or Restart_Filename_Base): This is the base filename of the restart files. For the given example parameter value, each processor will write a restart file restart.N, where N is the rank number of the processor. (Default: restart).

SnapshotFileBase (or Snapshot_Filename_Base): From this string, the name of the snapshot files is derived by adding an underscore, and the number of the snapshot in a 3-digits format. If NumFilesPerSnapshot>1, each snapshot is distributed into several files, where a group of processors writes simultaneously to a file. In this case, the filenames are also supplemented with a tailing .n, where n designates the number of the file in the group of files representing the snapshot. (Default: snapshot).

OutputListOn (or Use_Tabulated_Snapshot_Times): A value of 1 signals that the output times are given in the file specified by OutputListFilename. Otherwise output times are generated automatically in the way described below. (Default: 0, so this if off by default).

OutputListFilename (or Snapshot_Times_Table_Filename): This specifies the name of a file that contains a list of desired output times. If OutputListOn is set to 1, this list will determine the times when snapshot-files are produced. The file given by OutputListFilename should just contain the floating point values of the desired output times in plain ASCII format. The times do not have to be ordered in time. Output times that are in the past relative to the current simulation time will be ignored. (Ignored if OutputListOn is set to 0).

NumFilesPerSnapshot (or Number_of_Files_per_Snapshot): The code can distribute each snapshot onto several files. This leads to files that are easier to handle in case the simulation is very large, and also speeds up I/O, because these files can be written in parallel. The number of processors must be equal or larger than NumFilesPerSnapshot, because each snapshot file will hold the data of a group of processors. Optimum I/O throughput is reached if the number of processors is equal to, or a multiple of NumFilesPerSnapshot, and if NumFilesWrittenInParallel is equal to NumFilesPerSnapshot or a subdivision of it. With NumFilesPerSnapshot=1 it is possible to write all particle data in just one snapshot file. (Default: 1).

NumFilesWrittenInParallel (or Number_of_Files_Written_in_Parallel): This parameter determines how many files may be written (or read) simultaneously by the code when generating (reading) snapshot files, or when writing (reading) of restart files. Because each processor writes its own restart file, it is in principle possible to do this fully in parallel, with each processor writing their file at the same time. However, for big runs, say with 512 processors, a very large number of simultaneous I/O requests for large files may cause severe problems for the operating system, since the number of available disk drives on the file server will likely be smaller than the number of processors. As a result, the I/O system may get congested because these requests cannot all be served efficiently at the same time. In order to avoid such a congestion and to reach an optimum overall throughput of the I/O subsystem, it is often better not to write too many files simultaneously. This can be achieved with the parameter NumFilesWrittenInParallel, which restricts the number of files processed concurrently during restart or snapshot file reading/writing. Note that NumFilesWrittenInParallel must be equal or smaller than the number of processors used. (Default: 1).

%---- Output frequency  
TimeOfFirstSnapshot     0.1  % time (code units) of first snapshot  
TimeBetSnapshot         1.1  % time between (if OutputListOn=0), code units  
TimeBetStatistics       0.05 % time between additional statistics (e.g. energy)

TimeOfFirstSnapshot (or Simulation_Time_of_First_Snapshot): This variable selects the time for the first snapshot (if OutputListOn=0). Note that the time units are a=1/(1+z) for comoving integrations, while they are physical time for non-comoving integrations. (Ignored if OutputListOn is set to 1, otherwise defaults to the first time in the simulation set by TimeBegin).

TimeBetSnapshot (or ScaleFac_Between_Snapshots): If the user chooses not to specify a pre-set list of snapshot times, then after a snapshot has been written, the time for the next snapshot is determined in one of two ways. For non-cosmological simulations, recall that the time coordinate is just time, in physical units, and so the next snapshot time is determined by adding TimeBetSnapshot to the last snapshot time – i.e. the snapshots will be uniformly spaced in time with spacing equal to TimeBetSnapshot. For cosmological simulations (comoving integration), on the other hand, the time coordinate is the scale factor (a), and this represents the scale factor between snapshots, i.e. the next snapshot time is determined by multiplying the current snapshot time by this variable (hence the alternate name ScaleFac_Between_Snapshots). This will lead to a series of snapshots which are uniformly space in log(a). This variable is ignored if OutputListOn is set to 1.

TimeBetStatistics (or Time_Between_Internal_Diagnostic_Statistics): This determines the interval of time between two subsequent computations of the total energy of the system. This information is then written to the file given by EnergyFile. A first energy statistics is always produced at the start of the simulation at TimeBegin. (This has been deprecated to the DEVELOPER_MODE parameters, as this is used primarily for de-bugging purposes).

%---- CPU Run-Time and CheckPointing Time-Limits  
TimeLimitCPU              100000    % in seconds  
CpuTimeBetRestartFile     7200      % in seconds  
ResubmitOn        0  
ResubmitCommand   my-scriptfile

​
TimeLimitCPU (or MaxSimulationWallTime_in_Seconds): This is the CPU-time limit for the current run (one submission to the computing queue) in seconds. This value should be matched to the corresponding limit of the queueing system, if appropriate. The run will automatically interrupt itself and write a restart file, if 85% of this time has been consumed. The extra 15% is introduced to guarantee that there is always enough time left to safely finish the current time step and write the restart file. Note that this CPU time refers to the wall-clock time on one processor only. (Default: 1 day).

CpuTimeBetRestartFile (or Walltime_in_Seconds_Between_Restartfiles): This is the maximum amount of CPU-time (wall-clock time, in seconds) that may be used by the code before it writes a new restart file. With this parameter the code can hence be asked to write a restart file every once in a while. This is meant to provide a precautionary measure against hardware or software failures, in which case one can resume a simulation from the last set of restart files. In the above example, a restart file would be written automatically every 2 hours. The old set of restart files is renamed into a set of .bak files before the new files are written, so that there is some protection against a crash during the writing of the restart files itself (quite typically, this may happen due to disk-full errors, for example). (Default: 1 hour).

ResubmitOn (or Use_Automatic_Shell_Resubmission): This flag can be used to enable the automatic re-submission feature of the code (which is switched on for ResubmitOn=1). Note that many modern computing centers do not allow this kind of mechanism, and this will not work as a result. (Default: off).

ResubmitCommand (or Shell_Resubmission_Command): If ResubmitOn=1, the code will execute the given command (“my-scriptfile” in this example) when it interrupts the run because the CPU-time limit is about to be exhausted. Usually, “my-scriptfile” should be set to the name of an executable script in which one places the necessary commands for resubmission of the job (with the 1-flag set for restart) to the computing queue. Note that this feature should be used with care – it is always advisable to supervise a running simulation closely. (Default: ignored if ResubmitOn is 0)

TimeStepping and Run Begin & End Times

%---- Desired simulation beginning and end times (in code units) for run  
MaxSizeTimestep         0.005   % in code units, set for your problem  
MinSizeTimestep         1.0e-12 % set this very low, or get the wrong answer

MaxSizeTimestep (or Maximum_Timestep_Allowed): This parameter sets the maximum timestep a particle may take. This needs to be set to a sensible value in order to protect against too large timesteps for particles with very small acceleration. Usually, a few percent up to tens of percent of the dynamical time of the system gives sufficient accuracy. For cosmological simulations, the parameter specifies the maximum value of change (over one timestep) in the natural logarithm of the scale factor a=1/(1+z) (equivalent to specifying the maximum allowed timestep for cosmological simulations as a fraction of the current Hubble time). A value of 0.025 is about the maximum for cosmological runs, but smaller values give higher accuracy. (Default: small fraction, of order 0.001 times the snapshot spacing or total run time, but should really be adjusted as needed for the problem being simulated.)

MinSizeTimestep (or Minimum_Timestep_Allowed): If a particle requests a timestep smaller than the specified value, the simulation will issue a warning message and force the timestep to be at least MinSizeTimestep. This can be used to force equal and constant timesteps for all particles. If STOP_WHEN_BELOW_MINTIMESTEP is set in Config.sh, the code will quit instead when this occurs. (Default: smallest allowed by depth of number-of-timesteps that the long int register can hold).

%---- Maximum and minimum timesteps allowed  
TimeBegin           0.01      % Beginning of the simulation  
TimeMax             1.0       % End of the simulation

TimeBegin (or Initial_Simulation_Time): This initialises the time variable of the simulation when a run is started from initial conditions (in internal units). If comoving integration is selected (ComovingIntegrationOn=1), the time variable is the dimensionless expansion factor a itself, i.e. TimeBegin= a = 1/(1 + zstart). (Default: must be set manually for a cosmological simulation, but for non-cosmological simulations will default to initial time = 0).

TimeMax (or Final_Simulation_Time): This marks the end of the simulation. The simulation will run up to this point, then write a restart-file, and a snapshot file corresponding to this time (even if the time TimeMax is not in the normal sequence of snapshot files). If TimeMax is increased later on, the simulation can be simply continued from the last restart-file. Note that this last snapshot-file will then be overwritten, since it was a special dump out of the normal sequence. For comoving integrations, the time variable is the expansion factor, e.g. TimeMax=1.0 will stop the simulation at redshift z=0. Must always be set (no defaults).

Units & Cosmological Parameters

%---- System of units  
UnitLength_in_cm         3.085678e21    % 1.0 kpc/h  
UnitMass_in_g            1.989e43       % 1.0e10 solar masses/h  
UnitVelocity_in_cm_per_s 1e5            % 1 km/sec  
UnitMagneticField_in_gauss  1.0         % 1 gauss  
GravityConstantInternal  0              % calculated by code if =0

UnitLength_in_cm (or UnitLength_in_centimeters): This sets the internal length unit in centimeters. For cosmological simulations, the convention is to express these in “little-h” units (i.e. the value you set corresponds to cm$/h$, where \(H_{0} = 100\,h\) km/s/Mpc, so that the code’s internal representation is \(h\)-independent). The example value 3.085678e21 sets the length unit to 1.0 kpc$/h$ (i.e. 1 kpc when \(h=1\)). For non-cosmological simulations the \(h\) factor is irrelevant (set HubbleParam=1) and this is simply the length unit in cm. (Default: 1, only for simulations which do not need any dimensions).

UnitMass_in_g (or UnitMass_in_grams): This sets the internal mass unit in grams (with the same little-\(h\) convention as above for cosmological runs). The example value 1.989e43 sets the mass unit to \(10^{10}\,M_{\odot}\,h^{-1}\). (Default: 1, only for simulations which do not need any dimensions).

UnitVelocity_in_cm_per_s (or UnitVelocity_in_centimeterspersecond): This sets the internal velocity unit in cm/sec. The above choice is convenient – it sets the velocity unit to km/sec. Note that the specification of UnitLength_in_cm, UnitMass_in_g, and UnitVelocity_in_cm_per_s also determines the internal unit of time. The definitions made above imply that in internal units the Hubble constant has a numerical value independent of h and hence HubbleParam. For the numerical examples above, the Hubble constant has always the value 0.1 in internal units, independent of h, and the Hubble time is always 10 in internal units, with one internal time unit corresponding to 9.8e8 yr/h. However, you are free to choose a different system of units if you like. Note also that this implies that for purely gravitational dynamics, the code will not need to know the value of “h” at all. HubbleParam is kept in the parameterfile only because additional physics in the hydrodynamical sector may require it. (Default: 1, only for simulations which do not need any dimensions).

UnitMagneticField_in_gauss (or UnitMagneticField_in_Gauss_for_ICSnapshotIO): This sets the internal magnetic field unit in Gauss. Needed to be able to read/write input and output files. While actually working, the code will use the appropriate matching units for code pressure, etc. (Default = \(\sqrt{4\pi}\), only for simulations which do not need any dimensions).

GravityConstantInternal (or GravityConstant_inCodeUnits): The numerical value of the gravitational constant G in internal units depends on the system of units you choose. For example, for the numerical choices made above, G=43007.1 in internal units. For GravityConstantInternal=0 the code calculates the value corresponding to the physical value of G for you. Sometimes, you might want to set G yourself. For example, by specifying GravityConstantInternal=1, UnitLength_in_cm=1, UnitMass_in_g=1, and UnitVelocity_in_cm_per_s=1, one obtains a ‘natural’ system of units. Note that the code will nevertheless try to use the ‘correct’ value of the Hubble constant even in this case, so you should always set GravityConstantInternal=0 in cosmological integrations or any problem with gravity and real units, as opposed to some special numerical/code internal units (those are the kinds of problems this flag is designed for). (Default: 0)

%---- Cosmological parameters  
ComovingIntegrationOn 1 % is it cosmological? (yes=1, no=0)  
BoxSize               10000.0 % in code units  
Omega_Matter          0.27 % =0 for non-cosmological  
Omega_Lambda          0.73 % =0 for non-cosmological  
Omega_Baryon          0.044 % =0 for non-cosmological  
OmegaRadiation        0 % =0 for non-cosmological  
HubbleParam           0.7  % little 'h'; =1 for non-cosmological runs

ComovingIntegrationOn (or Cosmological_Simulation_On): This flag enables or disables comoving integration in an expanding universe. For ComovingIntegrationOn=0, the code uses plain Newtonian physics with vacuum or periodic boundary conditions. Time, positions, velocities, and masses are measured in the internal system of units, as specified by the selected system of units. For ComovingIntegrationOn=1, the integration is carried out in an expanding universe, using a cosmological model as specified by Omega0, OmegaLambda, etc. In this cosmological mode, coordinates are comoving, and the time variable is the expansion factor itself. If the code has not been compiled with the BOX_PERIODIC makefile option, the underlying model makes use of vacuum boundary conditions, i.e. density fluctuations outside the particle distribution are assumed to be zero. This requires that your particle distribution represents a spherical region of space around the origin. If BOX_PERIODIC is enabled, the code expects the particle coordinates to lie in the interval [0, BoxSize]. (Default: 0, i.e. non-cosmological).

BoxSize (or Box_Size_In_Code_Units): The size of the periodic box (in code units) encompassing the simulation volume. This parameter is only relevant if the BOX_PERIODIC makefile option is used. Particle coordinates need to lie in the interval [0, BoxSize] in the initial conditions file. Ignored if no code settings require a defined box size

Omega_Matter (or Omega0): Cosmological matter density parameter in units of the critical density at z=0. Generally relevant only for comoving integration. (Default in non-cosmological simulations: 0, will not influence results in this case).

OmegaLambda (or Omega_Lambda): Cosmological vacuum energy density (cosmological constant) in units of the critical density at z=0. Relevant for comoving integration, but can in principle be set for a non-periodic box to reflect uniform expansion. For a geometrically flat universe, one has Omega0 + OmegaLambda = 1. For simulations in Newtonian space (non-cosmological, non-expanding), this parameter has to be set to zero. (Default in non-cosmological simulations: 0, will not influence results in this case).

OmegaBaryon (or Omega_Baryon): Baryon density in units of the critical density at z=0. Relevant only for comoving integration. (Default in non-cosmological simulations: 0, will not influence results in this case).

OmegaRadiation (or Omega_Radiation): Radiation energy density in units of the critical density at z=0. Relevant only for comoving integration. (Default: 0, will not influence results in this case).

HubbleParam (or Hubble_Param_Little_h): Value of the Hubble constant in units of 100 km/s/Mpc. This is only used in cosmological integrations when conversions to cgs units are required (e.g. for radiative cooling physics). Note however that the value of HubbleParam is not needed (in fact, it does not enter the computation at all) in purely collisionless simulations. This is because of the definition of GIZMO’s system of units, which eliminates an explicit appearance of the value of the Hubble constant in the dynamical equations. (Default in non-cosmological simulations: 1, will not influence results in this case).

Memory & Load-Balancing (Domain) Parameters

%----- Memory allocation  
PartAllocFactor   2.0   % particle-load imbalance the decomposition may reach

WorkingMemoryPoolSize (or Working_Mem_Pool_Per_Task_in_MB): Sets the size, in megabytes per MPI task, of the working memory pool: a single block reserved at start-up that the code hands short-lived allocations out of and returns them to in reverse order. Reserving it once avoids repeated large allocations during the run, and it is accounted, so exhausting it produces a message naming what asked for the memory rather than a crash. It is not a limit on the task’s total memory: the particle data, the gravity tree, the buffers used to exchange data between tasks, and any memory used on an accelerator are allocated separately and are not counted against it. You should not normally set this. The code works out a size from the things that actually draw on the pool, once it knows the particle count, and prints the size it chose. Set it only to override that deliberately – for example if a run stops with a message naming this parameter, which means something needed more of the pool than the code predicted. (Older parameter files called this MaxMemSize; that name is no longer recognised, and the code says so at start-up.)

PartAllocFactor (or ParticleNumberMemoryImbalance_Limit): The largest particle-load imbalance the domain decomposition is allowed to produce, as a multiple of the balanced count: a task may be assigned up to PartAllocFactor times TotNumPart/NTask particles. Some imbalance has to be allowed, because insisting on an equal particle count per task would force a poor work-load balance. Values below 1 leave the decomposition with no solution and the run will stop. Bigger is not better. This is a ceiling on what a task may be assigned; it is not a memory reserve, and room for particles imported from neighbouring tasks is no longer taken out of it – that is measured per task from the imports that actually occur, and added on top. Raising it therefore buys no room for imported data, while it does inflate the gravity tree’s node arrays and the decomposition’s own limits, so a large value costs memory and gains nothing. Values a little above 1 are usually right; the default is 2. The case for setting it higher is a problem whose particle count grows substantially as it runs, through cell splitting or particle spawning.

CommChunkSize (or MPI_Comm_Chunk_Size_in_MB): The size, in megabytes, of the chunk the code sends particle data between tasks in – when writing and reading snapshot and initial-conditions field blocks, in the general-purpose all-to-all exchange, and in the neighbour loops’ communication rounds. A larger chunk means fewer, larger rounds; a smaller one means more, smaller rounds. It trades communication rounds against transient memory, and cannot by itself cause a failure. You should not normally set this: the code works out a size from what a round has to hold. (Older parameter files called this BufferSize; that name is no longer recognised, and the code says so at start-up.)

%---- Rebuild domains, and the gravity tree, when >this fraction of particles active  
DomainBuild_ActiveFraction   0.005  % 0.0005-0.05, dept on core+particle number  
TreeRebuild_ActiveFraction   0.005  % defaults to DomainBuild_ActiveFraction

DomainBuild_ActiveFraction (or TreeDomainUpdateFrequency): Neither the domain decomposition nor a full tree construction is carried out every single timestep; instead, properties of tree nodes can be dynamically updated as needed. This can improve performance a great deal when many timesteps advance only a very small fraction of particles (below a percent or so). This parameter says how often the particles are redistributed between tasks: a value of 0.05, for example, decomposes on any step where at least \(0.05\,N\) particles are active, where N is the total particle number. A decomposition always builds a new tree with it. If you find that imbalance (bad load-balancing) dominates runtime (see the cpu.txt file), try decreasing the value here: values from 0.0005–0.05 can work well. (Default: 0.05, but we stress this is rather ad-hoc and users should experiment with the choice.)

TreeRebuild_ActiveFraction: The same thing for the gravity tree alone – the tree is rebuilt from scratch on any step where at least this fraction of the system is active, keeping the existing decomposition. A value of zero rebuilds the tree every timestep. Redistributing particles is a cost and load-balancing question, while how long a tree may be reused before its geometry stops describing where the particles actually are is an accuracy one, which is why the two are set separately: lower this one to buy accuracy without paying for a decomposition you have no other use for. Because a decomposition always brings a new tree with it, setting this above DomainBuild_ActiveFraction has no effect. (Default: whatever DomainBuild_ActiveFraction ends up being, which reproduces the older behaviour where a single parameter controlled both. A parameterfile written for the older code, which set only TreeDomainUpdateFrequency, therefore behaves exactly as it did before.)

GravityHostWalkBelowActive: Advanced performance tuning (optional). On any step where an MPI task has fewer than this many active elements needing a gravity-tree force, that task uses the host gravity-tree walk; at or above it, the task uses the parallel device walk (the GPU in GPU builds, all OpenMP threads in CPU-only builds). With few active elements the host walk is the cheaper route, with many the device walk is. Forces are the same on either route. The count is per task, not summed over tasks. Setting 0 disables this count-based routing. (Default: 1000.)

TreeUpdateHostBelowActive: Advanced performance tuning (optional), the companion of GravityHostWalkBelowActive for the dynamic update of the gravity tree – the step-to-step update of node properties that lets the tree be reused between rebuilds (see TreeRebuild_ActiveFraction). On any step where an MPI task has fewer than this many elements due a force update, the update runs on the host, which advances only the nodes the active elements actually reach; at or above it, the task uses the parallel device update (the GPU in GPU builds, all OpenMP threads in CPU-only builds), which advances either every node in the tree or only the nodes the update touches (see TreeUpdateFullSweep_ActiveFraction). The two are set separately because the costs that decide them are different, so their best values need not coincide. Setting 0 disables the count-based routing; the host is still used for the rest of any timestep in which the host has already advanced some nodes, which the device update cannot take over. (Default: 1000.)

TreeUpdateFullSweep_ActiveFraction: Advanced performance tuning (optional), for the device form of the dynamic tree update (see TreeUpdateHostBelowActive). Before it propagates a step’s momentum kicks up the tree, the device update must bring the nodes it is about to touch up to the current time. When the fraction of this MPI task’s elements being updated is at least this value, it does so by advancing every node in the tree, which also lets every walk that follows use the whole tree without advancing anything itself; below it, it advances only the chains of nodes above the updated elements and the top-level tree, and the walks that follow advance whatever else they open as they read it. The fraction is the larger of the elements whose kicks are being propagated and those due a force update in the coming step, divided by the task’s element count. Results are the same either way to round-off; this only moves work. Setting 0 always advances every node, which can pay when nearly every step updates a large part of the tree. A value above 1 advances only the chains on every update, except that a chain list too long for its buffer is answered by advancing every node; this is the recommended setting, because advancing every node right after a tree build is expensive on some GPU systems (the node memory has to move back to the device) while the chains cost the same as a full advance on ordinary steps. (Default: 2.)

Hydro Parameters

%---- (Optional) Initial hydro temperature & temperature floor (in Kelvin)  
InitGasTemp           300   % set by IC file if =0  
MinGasTemp             10.  % don't set <10 in explicit feedback runs, otherwise 0

InitGasTemp (or Initial_Gas_Temperature): This sets the initial gas temperature in Kelvin when initial conditions are read. If the temperature is below 1e4 K, a mean molecular weight corresponding to neutral gas of primordial abundance is assumed, otherwise complete ionisation is assumed. However, the gas temperature is only set to a certain temperature if InitGasTemp>0 and if at the same time the temperature of the gas particles in the initial conditions file is zero, otherwise the initial gas temperature is given by the value stored in the IC file. (Default: 0).

MinGasTemp (or Minimum_Gas_Temperature): The minimum allowed gas temperature in Kelvin (this is converted by the code to a minimum thermal energy per unit mass assuming the mean molecular weight of neutral gas). This may be set to zero in principle, but often it is desirable to prevent the gas from becoming too cold, e.g. for resolution reasons or because of lower limits in the implemented cooling function. (Default: 0).

%---- Hydro reconstruction (kernel) parameters  
DesNumNgb               32      % domain-reconstruction kernel number: 32 standard, 60-114 for quintic  
MaxKernelRadius                 1.0e10  % maximum gas kernel length (some very large value to prevent errors)  
MinGasKernelRadiusFractional    0       % minimum kernel length relative to gas force softening (<= 1)  
%----- adaptive grav softening or other kernel-interactions for non-gas elements  
AGS_DesNumNgb             32  % neighbor number for calculating interaction kernel

DesNumNgb (or Effective_Kernel_NeighborNumber): This is the desired “effective” number of particle kernel neighbors, used for both the meshless volume partition, SPH “kernel smoothing”, adaptive gravitational force softening, and local interactions of non-standard particles (e.g. dust). We say “effective” because, as is standard, the number is defined to be continuous based on the local number density of particles at the particle/cell position and its size, rather than the discrete actual number of particles inside a sphere (a discontinuous and therefore numerically problematic quantity). Should the value get outside the range plus/minus MaxNumNgbDeviation from DesNumNgb, the code will readjust such that the number of neighbours is again in this range. If the code is run in one or two dimensions, smaller numbers are needed: for 32 in 3D, use ~4–8 in 1D, and ~14–20 in 2D, if youre using the cubic spline kenrel, for example, but this needs to be adjusted based on the kernel function adopted. (Default: will adopt a guess based on the kernel chosen if you do not specify, but this will almost certainly not be optimal in terms of either errors or performance. There are very specific values for certain kernel functions which can give much more accurate results by minimizing certain errors, so it is always recommended to set this manually if you know what you are doing).

NeighborLoopModeBThresholdSum / NeighborLoopModeBThresholdMax: Advanced performance/dispatch tuning (optional). The neighbor-loop machinery and the hydro corridor pick between a batched “Mode A” (build a shared neighbor list) and a request-driven “Mode B” per step based on how many particles are active; Mode B is chosen only when the globally summed active count is at or below NeighborLoopModeBThresholdSum and the per-rank maximum active count is at or below NeighborLoopModeBThresholdMax. If unset (the default), each loop uses its own built-in threshold. Setting a value of 0 disables Mode B on the adaptive paths (forcing Mode A); setting both to a very large value approximates forcing Mode B, but does not override compile-time constraints (e.g. TRANSPORT_SUBCYCLE always uses Mode A) or correctness/safety guards. (Default: unset = use per-loop defaults. –1 also means unset; values below –1 are rejected.)

MaxKernelRadius (or Maximum_KernelRadius_CodeUnits): The maximum allowed value of the kernel length (in code units). This is here only to prevent very rare cases where an unusual particle distribution (for example, an isolated gas particle in the far corner of a cosmological “zoom in” box which has escaped the high-resolution region) lead to the code spending forever iterating to try to find the “right” number of neighbors across the box. Set it to something comparable to a decent fraction of BoxSize, or an arbitrarily large value if you aren’t worried about this type of error (usually not a problem). (Default: no upper limit).

MinGasKernelRadiusFractional (or Minimum_Gas_KernelRadius_RelativetoSoftening): This sets the minimim allowed kernel length in units of the gravitational softening length (specified below) of the gas particles. The kernel length will be prevented from falling below this value (the number of neighbors will be allowed to exceed DesNumNgb). This can be zero (recommended for problems -without- gravity), but in problems with gravity, this “hydro resolution” is meaningless (because there is no gravity on these scales). Don’t be misled by papers claiming their “gas resolution” is much better than their “gravity resolution”, this exists only for numerical convenience! If adaptive gravitational softening lengths for gas or all particles (ADAPTIVE_GRAVSOFT_FORGAS or ADAPTIVE_GRAVSOFT_FORALL or ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION) is set, this parameter should be set=1. If fixed softenings are used, you can set it to a value ~0.3–0.5, but again we caution the “real” resolution is the gravitational resolution.

Whatever the value of this parameter, the term that sets the minimum actual code-allowed hydro kernel size (this parameter multiplied by the gas force softening parameter) must be set sufficiently small so that you do not end up with a very large number of particles (much more than the intended target DesNumNgb) inside the kernel. In SPH, if the hydro kernel is much larger than the inter-neighbor separation, this is fine in some ways (since the kernel is just ‘smoothing’ some properties), but it can trigger the pairing instability if the kernel is not chosen to match the neighbor number achieved. In MFM/MFV-type methods, the result is even worse: in these methods the code computes fluxes between neighbors by using a mesh-free volume partition between them (which you can think of as a kernel-based approximant of a Voronoi tesselation). But if the code cannot adapt the kernel size (as it normally tries to do) to enclose the desired DesNumNgb (but instead is ‘fixed’ at a given minimum size), this tesselation will be forced to the wrong answer, and so the fluxes (still computed assuming a valid tesselation between neighbors) will also be wrong. In either case, this can lead to very bad results, so don’t enforce large minimum gas kernel sizes. (Default: 0, unless using adaptive force softening, then this defaults to = 1).

AGS_DesNumNgb (or AdaptGravSoft_Effective_NeighborNumber): If you enable adaptive gravitational softenings, or SIDM modules, or grain collision modules are used – anything which requires explicit kernel-wise interactions between non-gas particles, the softenings are set on-the-fly at all times based on the inter-particle distance. AGS_DesNumNgb and AGS_MaxNumNgbDeviation function exactly like DesNumNgb and MaxNumNgbDeviation, they are simply applied to particles whose type is not gas, to obtain their appropriate kernel lengths. In general, AGS_DesNumNgb should be set to the same value as DesNumNgb, but it can (in some special cases) be somewhat smaller or larger (e.g. if, as is required sometimes for SPH-mode simulations, DesNumNgb very large but the adaptively-softened particles do not include gas) for the sake of speed or accuracy. However the same kernel function is used for both, so avoiding numerical issues like the pairing instability will generally require these are similar to one another. (Default: if not specified the code will try to adopt a reasonable default value based on the kernel provided, akin to DesNumNgb, but see the notes above for that, as the same caveats apply.)

Note on configuring adaptive softening: If you use any of ADAPTIVE_GRAVSOFT_FORGAS, ADAPTIVE_GRAVSOFT_FORALL, or ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION, you must carefully set three parameterfile values together: (1) Softening_TypeX sets the minimum allowed softening for each particle type (the softening will never go below this); (2) MaxKernelRadius sets the maximum allowed softening (it will never exceed this); and (3) AGS_DesNumNgb controls the target neighbor number used to set the adaptive softening for non-gas particles. If any of these are misconfigured (e.g., minimum too large, maximum too small, or neighbor number inappropriate), the adaptive softening may not behave as intended.

Gravity Parameters

Softening…: These are the gravitational force-softening parameters. Note that, to ensure clarity but also historical compatibility with tools and scripts written for GADGET and AREPO, you can use either of two different name conventions in the parameter file for the gravitational softenings. Examples of both are below.

The preferred ‘new’ convention, which more generally reflects modern useage of GIZMO, is below: 

%---- Gravitational softening lengths, in code units  
%----- Softening lengths per element type. If ADAPTIVE_GRAVSOFT is set, these  
%-------- are the minimum softening allowed for each type -------  
%-------- (units are co-moving for cosmological integrations)  
Softening_Type0     0.001    % element/particle type 0 [gas]  
Softening_Type1     0.020    % element type 1 [e.g. high-res dark matter]  
Softening_Type2     0.150    % element type 2 [e.g. dummy collisionless]  
Softening_Type3     0.500    % element type 3 [e.g. grains/PIC particles]  
Softening_Type4     0.001    % element type 4 [e.g. stars, in galaxy sims]  
Softening_Type5     0.001    % element type 5 [e.g. black holes/sinks]

Recall, the code distinguishes between different particle/element types. Each type has a different gravitational softening set here. As far as gravitational physics alone is concerned (assuming the code is running in its default mode), the types are just arbitrary tags: but of course other physics will treat these differently and they may have very different resolution, hence different softenings. Note the user must always specify all of these, because at compilation and parameterfile read-in there is no way for the code to know which types may at some point become active (many modules convert between various types). But if a particle type never appears in your simulation, you can set the corresponding value to anything (e.g. 0) with no ill effects.

Gravity is softened with a spline, as outlined in the code paper. The softenings quoted here all refer to the roughly equivalent Plummer softening length. Note that for the spline that is actually used, the force will be exactly Newtonian beyond 1/KERNEL_FAC_FROM_FORCESOFT_TO_PLUMMER (value defined in the code in kernel.h, =2.8 for example for a cubic spline kernel) times this value. The code will tell you exactly what this factor is on startup in the stdout file. The softening lengths are in internal (code) length units. For cosmological simulations, that means these refer to co-moving units.

If adaptive softening is employed, the softening lengths above still need to be set: in this case, these represent the minimum allowed softening for a given particle type. This is done to prevent the code from going beyond user-specified unphysical resolution limits (e.g. infinitely high densities in self-gravitating collapse).

If your simulation is cosmological, you must also specify the corresponding Softening_TypeX_MaxPhysLimit for each softening type X. 

%---- if these are set in cosmo runs, SofteningX switches from comoving to physical  
%------- units when the comoving value exceeds the choice here  
%------- (these are ignored, and *only* the above are used, for non-cosmo runs)  
Softening_Type0_MaxPhysLimit    0.0005  % e.g. switch to 0.5pc physical below z=1  
Softening_Type1_MaxPhysLimit    0.010  
Softening_Type2_MaxPhysLimit    0.075  
Softening_Type3_MaxPhysLimit    0.250  
Softening_Type4_MaxPhysLimit    0.0005  
Softening_Type5_MaxPhysLimit    0.0005

With these defined, the code will switch to a fixed physical softening length (still in code units, but not co-moving) when the co-moving length specified by the normal softening above exceeds the MaxPhysLimit specified here. In practice, the code uses MIN( Softening_TypeX/(1+z) , Softening_TypeX_MaxPhysLimit ) as the physical softening for particle of type X at a time corresponding to redshift z. Again, if the softening is adaptive, this represents the minimum allowed value at a given redshift. If the run is non-cosmological, these MaxPhysLimit parameters are ignored, and do not need to be specified in the parameterfile at all.

Alternatively, for historical reasons, you can choose to use the naming convention for the force-softening from GADGET and AREPO. This looks like the following, in your parameterfile:

%---- Gravitational softening lengths  
%----- Softening lengths per element type. If ADAPTIVE_GRAVSOFT is set, these  
%-------- are the minimum softening allowed for each type -------  
%-------- (units are co-moving for cosmological integrations)  
SofteningGas       0.001    % gas (particle type=0) (in co-moving code units)  
SofteningHalo      0.020    % dark matter/collisionless particles (type=1)  
SofteningDisk      0.150    % collisionless particles (type=2)  
SofteningBulge     0.500    % collisionless particles (type=3)  
SofteningStars     0.001    % stars spawned from gas (type=4)  
SofteningBndry     0.001    % sink particles (if active), or collisionless (type=5)  
%---- if these are set in cosmo runs, SofteningX switches from comoving to physical  
%------- units when the comoving value exceeds the choice here  
%------- (these are ignored, and *only* the above are used, for non-cosmo runs)  
SofteningGasMaxPhys       0.0005  % e.g. switch to 0.5pc physical below z=1  
SofteningHaloMaxPhys      0.010   % [type=1]  
SofteningDiskMaxPhys      0.075   % [type=2]  
SofteningBulgeMaxPhys     0.250   % [type=3]  
SofteningStarsMaxPhys     0.0005  % [type=4]  
SofteningBndryMaxPhys     0.0005  % [type=5]

Here SofteningGas is exactly equivalent to Softening_Type0 described above. Likewise SofteningGasMaxPhys is identical to Softening_Type0_MaxPhysLimit, SofteningHalo is identical to Softening_Type1, etc. The names ‘Gas’ (Type=0), ‘Halo’ (Type=1), ‘Disk’ (Type=2), ‘Bulge’ (Type=3), ‘Stars’ (Type=4), and ‘Bndry’ (Type=5), are just arbitrary conventions, still reflecting our predecessor GADGET–2’s origin as a tool to study colliding galaxies. In fact the names are interchangeable: you can mix-and-match whichever naming conventions you like (so long as you only define the relevant variable once per parameterfile).

Parameters of Optional Physics Modules (Examples)

There are many different, optional modules for additional physics in the code. Many of these use some runtime parameters which should be included in the parameterfile. The role of those parameters and their recommended values depends on the module, and you should look at the code for each module you enable to decide which values are needed. In general, if a required parameter is not present, the code will quit on start-up with a clear error message specifying the missing parameter.

Here, we will list and describe some of these, but this will necessarily be an incomplete list and should not be taken as final. Consult the code for each module to determine what the parameters mean.

Additional Fluid Physics

%---- Magneto-Hydrodynamics Parameters (MAGNETIC on)  
%--- Initial B-Field Strengths (if MHD_B_SET_IN_PARAMS on, otherwise read from IC file)  
BiniX   1.0e-8    % initial B_x, in code units  
BiniY   1.0e-8    % initial B_y, in code units  
BiniZ   1.0e-8    % initial B_z, in code units

BiniX/BiniY/BiniZ (or B_initial_x, B_initial_y, B_initial_z): If MHD_B_SET_IN_PARAMS is turned on at compile-time, then when starting from an initial condition (not a restart), the B-fields of all gas are initialized to these X/Y/Z values (given here in Gauss, these will be converted to code units). If MHD_B_SET_IN_PARAMS is not active, the initial B-fields will be read from the ICs file instead (these values are ignored).

%--- Modified-Gradient div(B) correction (MHD_MODIFIED_GRADIENT on)  
ActiveFractionForMGSweep   0.0    % fraction of active cells required to trigger MG global solve (0=always, 1=never)

ActiveFractionForMGSweep: If MHD_MODIFIED_GRADIENT is enabled, this controls how frequently the global sparse-matrix div(B) correction is applied. At each sync-point, the code computes the fraction of all gas cells that are active at that timestep. If this fraction is greater than or equal to ActiveFractionForMGSweep, the full MG solve (via HYPRE BoomerAMG or the built-in SSOR-CG solver) is executed; otherwise the step is skipped and the cheaper CG-only correction is applied instead. Setting this to 0.0 (or omitting it entirely, which defaults to 0.0) applies the MG solve every sync-point, giving maximum div(B) control at the highest cost. Setting it to a value like 0.1 applies the global solve only on the larger timestep intervals (when ≥10% of cells are active), which can recover most of the accuracy benefit with substantially lower overhead in multi-timestep simulations.

%---- Thermal Conduction (CONDUCTION on)  
%--- set coefficient kappa [code units] or, if CONDUCTION_SPITZER on, multiplies value  
ConductionCoeff  1.0  % set/multiply conduction coefficient

ConductionCoeff: This defines the conductivity (\(\kappa\)) coefficient, for physical thermal conduction (CONDUCTION on). If CONDUCTION_SPITZER is on, the relevant coefficients are calculated physically; these are simply dimensionless multipliers than the code will adopt a default value = 1. If, for example, you run a non-MHD version of the calculation, people often set a multiplier \(<1\) to represent suppression by tangled magnetic fields. Of course, since the code correctly handles anisotropic conduction with MHD, you should really use that! If CONDUCTION_SPITZER is off, these are used directly (they should be given in code units: we use conductivities here, not diffusivities, so the units are mass / (time \(\times\) length). 

%---- Navier-Stokes Viscosity (VISCOSITY on)  
%--- set coefficients eta,zeta [code units] or, if VISCOSITY_BRAGINSKII on, multiplies value  
ShearViscosityCoeff     1.0     % set/multiply shear viscosity coefficient  
BulkViscosityCoeff      1.0     % set/multiply bulk viscosity coefficient

ShearViscosityCoeff/BulkViscosityCoeff: These define the shear (\(\eta\)) and bulk viscosity (\(\zeta\)) coefficients, for physical Navier-Stokes viscosity (VISCOSITY on). If VISCOSITY_BRAGINSKII is on, the relevant coefficients are calculated physically; these are simply dimensionless multipliers. Again when this is set, the default value the code will adopt (if this is not set) is =1. If, for example, you run a non-MHD version of the calculation, people often set a multiplier \(<1\) to represent suppression by tangled magnetic fields. Of course, since the code correctly handles anisotropic viscosity with MHD, you should really use that!. If VISCOSITY_BRAGINSKII is off, these are used directly (they should be given in code units: we use dynamic viscosities here, so the units are mass / (time \(\times\) length). 

%---- Turbulent Diffusion Top-level Switch (TURB_DIFFUSION on)  
TurbDiffusionCoefficient  1.0   % Normalizes diffusion rates, should be ~1

TurbDiffusionCoefficient: Multiplies the dimensionless diffusion coefficient for the Smagorinski turbulent eddy diffusion subgrid-scale model (relative to the value ‘recommended’ by Smagorinski in three dimensional problems). This multiplies all the diffusion rates set by TURB_DIFFUSION_X compile-time flags. Default =1.

%--- Cosmic Ray + Gas Fluids (COSMIC_RAY_FLUID on)  
CosmicRayDiffusionCoeff     690    % diffusion coefficient of the single-bin CR model, or normalization for spectral fit (for ~GeV CRs) in code units  
CosmicRay_SNeFraction       0.1     % fraction of SNe ejecta kinetic energy into cosmic rays (~10%)

CosmicRayDiffusionCoeff: Set the diffusion coefficients for cosmic ray diffusion/streaming (COSMIC_RAY_FLUID on), assuming you are using the fixed-scattering-rate models CRFLUID_DIFFUSION_MODEL=0, as opposed ot the models where the CR scattering rates are set dynamically according to different physical models. For the single-bin CR models, this is just the diffusion (note that the CRs can also stream at the Alfven speed, as described in Chan et al. and Hopkins et al. methods papers on this as noted in the Config file descriptions for these flags). For the multi-bin (resolved) models, this normalizes the diffusivity and scattering rate at a rigidity of 1 GV. The value should be in code units. Recall that the observationally-constrained effective anisotropic (parallel) diffusion coefficient measured at the Solar circle from a wide variety of data (detailed spectra, spallation products, radioactive products, gamma-ray emission, ionization, synchrotron) is 3e29 cm2/s, in cgs.

CosmicRay_SNeFraction (or CosmicRay_SNeEnergyFraction): If COSMIC_RAY_FLUID and GALSF_FB_MECHANICAL are both on with the default FIRE physics implementation active as well, then a fraction (typically calculated and observationally inferred to be \(\sim 0.1\)) of the initial supernova kinetic energy of each explosion is deposited into cosmic rays.

%--- Sub-grid LEBRON cosmic-ray model (COSMIC_RAY_SUBGRID_LEBRON on)  
CosmicRay_Subgrid_Kappa_0       120     % effective CR diffusivity [code length * code velocity]  
CosmicRay_Subgrid_Vstream_0     20      % effective CR streaming speed [code velocity]

CosmicRay_Subgrid_Kappa_0/CosmicRay_Subgrid_Vstream_0: If COSMIC_RAY_SUBGRID_LEBRON is on, these set the constant effective diffusion coefficient \(\kappa_{0}\) and effective streaming speed \(v_{\rm st,0}\) of the steady-state, spherically-symmetric CR profile assumed around each source (Hopkins et al. arXiv:2211.05811). The CR energy density from each source falls off as \(\propto 1/[4\pi\,r\,(\kappa_{0} + v_{\rm st,0}\,r)]\), truncated beyond the maximum distance the CRs could have propagated in the source age; the same two values set the local loss-attenuation factor. Both are in code units (physical): \(\kappa_{0}\) in code length times code velocity, \(v_{\rm st,0}\) in code velocity. Defaults are 120 and 20, the values calibrated in the FIRE simulations, which assume code units of kpc/h and km/s; rescale them if your units differ.

%---- Dust-Gas Mixtures (GRAIN_FLUID on)  
Grain_Internal_Density      1.0     % internal/material density of grains in g/cm^3  
Grain_Size_Min              1.e-6   % minimum grain size in cm  
Grain_Size_Max              1.e-4   % maximum grain size in cm  
Grain_Size_Spectrum_Powerlaw  0.5   % power-law distribution of grain sizes (dm/dlnr~r^x)  
%-------------- parameters for grain-grain collisions/scattering/sticking/shattering (GRAIN_COLLISIONS on)  
Grain_InteractionRenormalization 1     % coefficient to re-normalize cross-section (otherwise hard-sphere assumed; default=1)  
Grain_InteractionVelocityScale 0       % velocity [code units]: if>0, cross-section scales ~1/(1+(v/v0)^4) above this "v0"  
Grain_DissipationFactor 0              % 0=elastic, 1=pure dissipative (fractional dissipation of kinetic energy in event)  
Grain_KickPerCollision  0              % velocity 'kick' [code units] per collision (this^2=specific energy/mass released)

Grain_Internal_Density: This specifies the internal or material density of the grains (all grains are assumed to have the same density), in cgs (\({\rm g\,cm^{-3}}\)). This (together with the code units and grain sizes) will determine the grain drag forces.

Grain_Size_Min/Grain_Size_Max: Minimum/maximum grain size (in cm). By default, the code will uniformly distribute grain “super-particles” in log-size between the minimum and maximum sizes. If you wish to hard-code grain sizes in the initial conditions, or modify the grain size spectrum generated upon startup of the simulation, you just need to modify this line in init.c (specifically look for the line containing All.Grain_Size_Min, which sets the variable P[i].Grain_Size used in the computations). If you set Grain_Size_Max less than or equal to Grain_Size_Min, the grains will all be assigned the same size (equal to Grain_Size_Min).

Grain_Size_Spectrum_Powerlaw: This specifies a desired power-law distribution of grain sizes. If Grain_Size_Max is less than Grain_Size_Min, or this is =0, this will not activate (the grains will be assigned the uniform size distribution as above, and whatever masses the user initially assigns). If this is set to a value not =0, then the grain super-particle masses will be re-weighted, so that the number of super-particles per logarithmic interval in mass is still constant, as described above, but the mass of each super-particle – i.e. the number of actual micro-physical grains it represents – will vary. This allows you to retain good numerical sampling of all grain sizes even if some grain sizes are very rare, proportionally speaking. Specifically, this flag assumes the initial dust super-particles all have equal mass, then re-weights them accordingly (conserving the total mass of dust, but not the mass-per-particle), so that the final distribution samples number of grains uniformly in log of grain size, and gives \(d M_{\rm grains}^{\rm total} / d\ln{R_{\rm grain}} \propto R_{\rm grain}^{\alpha}\), where \(\alpha=\) Grain_Size_Spectrum_Powerlaw. Note, for grains of constant internal (material) density, this is related to the usual grain size spectrum \(dN_{\rm grains}/d R_{\rm grain} \propto R_{\rm grain}^{-\beta}\) by \(\alpha=4-\beta\) (so, for example, the classic MRN size spectrum corresponds to Grain_Size_Spectrum_Powerlaw=0.5, as set in the example above).

Grain_InteractionRenormalization: If GRAIN_COLLISIONS is on, this allows you to arbitrarily renormalize the grain self-interaction cross section. By default, the code assumes hard sphere scattering, so \(\sigma_{\rm grain} = (\pi\,R_{\rm grain}^{2})/m_{\rm grain}\) (where \(m_{\rm grain}\) is the grain mass determined by the size and material density specified). This parameter simply multiples that (hence default \(=1\)). If you wish to insert a more complicated scaling for grain cross-sections, this is easily done in the function return_grain_cross_section_per_unit_mass in grain_physics.c.

Grain_InteractionVelocityScale: If GRAIN_COLLISIONS is on, this sets the grain self-interaction cross section dependence on velocity. Specifically, if this is set to be \(>0\), then the cross section is assumed to scale as \(\sigma(v) = \sigma_{0}/[1 + (v/v_{0})^{4}]\) where \(\sigma_{0}\) is the value given for hard-sphere scattering as described above, and \(v_{0}\) is the value of Grain_InteractionVelocityScale – i.e. this is given as a velocity in code units. Note by default we assume Rutherford-type scattering (for e.g. Coulomb interactions between charged grains) at high velocities (\(\propto v^{-4}\)): if desired, it is trivial to change this by editing the one line where this scaling appears in the function prob_of_grain_interaction in grain_physics.c. If the parameter is set \(<=0\), then it will have no effect (i.e. the scattering is velocity–independent).

Grain_DissipationFactor: If GRAIN_COLLISIONS is on, this sets the fractional degree of dissipation per interaction event. This interpolates between purely elastic scattering when equal to 0, to ‘hit-and-stick’ or purely momentum-conserving scattering (both particles move at the center-of-momentum, i.e. all kinetic energy in the center-of-mass frame is entirely dissipated, so this is the most dissipative possible) when equal to 1. For intermediate values, this multiplies the post-collision recoil velocity in the center-of-mass frame, so the center-of-mass frame kinetic energy is decreased by a factor of this squared.

Grain_KickPerCollision: If GRAIN_COLLISIONS is on, this sets the velocity of ‘kicks’ given to the grains in scattering events beyond elastic scattering. More specifically, this value (in code units) squared defines the specific energy (energy per unit mass) added to the kinetic energy of the scattering grains in the rest frame of the scattering event – this represents models where e.g. some of the chemical energy of the grains is liberated in collisions, sending pieces of grains moving off at higher velocities.

Note that there are a variety of other grain parameters that can be enabled if the user specifies certain config flags. As described in the GRAIN_RDI_TESTPROBLEM description, this enables 

%-------------- parameters for additional grain simulations (GRAIN_RDI_TESTPROBLEM on)  
Grain_Charge_Parameter             1  % dimensionless charge (3*q_grain/[4pi*c*R_grain^2*sqrt(rho_box)])  
Dust_to_Gas_Mass_Ratio          0.01  % dust/gas by mass: ~0.01 for all grains, for solar abundance  
Vertical_Grain_Accel             5e7  % vertical radiative accel in code units (cs^2/Lbox)  
Vertical_Grain_Accel_Angle         0  % angle (in deg) between grain accel and grav accel  
Vertical_Gravity_Strength        1e5  % vertical gravity in code units: = L_box/H_gravity_scale_length  
Grain_Q_at_MaxGrainSize          0.2  % Q normalization at maximum grain size  
Grain_Absorbed_vs_Total_Extinction 0  % absorbed (vs scattered) fraction

Grain_Charge_Parameter is the dimensionless grain charge parameter, \(\tilde{\phi}_{\rm max}\), from Lee et al. 2017MNRAS.469.3532L and Seligman et al 2019MNRAS.485.3991S, defined for the largest grains.

Dust_to_Gas_Mass_Ratio the mean dust to gas mass ratio of the box (grain super-particle masses will be renormalized assuming their initial total mass in the box is unity, to give this value - don’t use this unless your ICs are actually set up appropriately)

Vertical_Grain_Accel: the value of the acceleration from external forces (e.g. radiation) on dust grains (in code units), normalized to the largest grains (for simulations with live radiation-dust-magnetohydrodynamics, this normalizes the incident radiation flux to the parameter \(a_{0}\) as described in Hopkins et al. arXiv:2107.04608)

Vertical_Grain_Accel_Angle: an arbitrary angle in degrees between the direction of the grain acceleration and gravity (defining e.g. anti-parallel for this equal to zero)

Vertical_Gravity_Strength: the value of the externally imposed, constant analytic gravitational force, in code units

Grain_Q_at_MaxGrainSize: value of the dimensionless grain extinction efficiency \(Q\) (cross section relative to geometric, at the wavelength of interest), for the largest grains, if RT_OPACITY_FROM_EXPLICIT_GRAINS is set

Grain_Absorbed_vs_Total_Extinction: fraction of incident radiation absorbed (1-albedo), assumed for grains, if RT_OPACITY_FROM_EXPLICIT_GRAINS and RT_GENERIC_USER_FREQ are set.

%-------------- grain microphysics on GRAIN_FLUID super-particles (GRAIN_EVOLUTION on)  
GrainEvolution_StickingCoeff              1.0   % multiplies the coagulation velocity threshold, and is the sticking coefficient for condensation  
GrainEvolution_VelThreshFrag              0     % read but not currently used (set any value)  
GrainEvolution_VelThreshShat              0     % read but not currently used (set any value)  
GrainEvolution_ThermalSputteringScaling   1.0   % multiplies the thermal-sputtering erosion rate  
%-------------- promotion of grain super-particles to solid gas cells (GRAIN_FLUID_PROMOTION on)  
GrainPromotion_MassThresh_cgs        0.01   % grain super-particles with mass >= this (in g) are promoted  
GrainPromotion_DustGasRatioThresh    0      % if >0, also require local grain/gas density ratio >= this

GrainEvolution_StickingCoeff: If GRAIN_EVOLUTION is on (requires GRAIN_FLUID), this multiplies the species-dependent coagulation velocity threshold \(v_{\rm coag}\) for the pairwise collision outcomes (bits 1/2/4, which also need GRAIN_COLLISIONS): collisions with \(|\Delta v| < v_{\rm coag}\times\)GrainEvolution_StickingCoeff coagulate, faster ones (up to the shattering velocity) fragment. For condensation (bit 32) it is used directly as the sticking coefficient (values \(\le 0\) are replaced by 1). Set =1 for the species defaults. Required (no default).

GrainEvolution_VelThreshFrag/GrainEvolution_VelThreshShat: Read when GRAIN_EVOLUTION is on, and required in the parameterfile, but not used anywhere by the current code: the fragmentation and shattering thresholds come from the per-species defaults. Any value can be given.

GrainEvolution_ThermalSputteringScaling: If GRAIN_EVOLUTION thermal sputtering (bit 8) is on, multiplies the grain-radius erosion rate \(da/dt \propto -Y_{\rm sput}(T)\,n_{\rm H}\). This is the analogue of ThermalSputteringScaling in the ISM dust-chemistry model. =1 is nominal. Required (no default).

GrainPromotion_MassThresh_cgs: If GRAIN_FLUID_PROMOTION is on (requires GRAIN_FLUID; it auto-enables EOS_TILLOTSON, EOS_TYPES_DEFAULTGAS_AND_SOLIDS and IO_COMPOSITIONTYPE_NOT_IN_ICFILE), each active grain super-particle with mass at or above this value (given in grams, converted to code units at startup) is converted into a Type=0 solid cell with a Tillotson CompositionType. Under GRAIN_EVOLUTION the type follows the grain’s dominant composition (silicate to olivine, carbon to basalt, iron to iron, ices to ice); otherwise it is olivine. A threshold of 0 promotes every grain that passes the density-ratio test. Promotion stops if the gas block would exceed MaxPartGas. Required (no default); can be changed on restart.

GrainPromotion_DustGasRatioThresh: If >0, a grain super-particle is promoted only if, in addition to the mass criterion, the local grain density estimate (neighbour number times super-particle mass over the kernel volume) divided by the local gas density is at least this value. Set =0 to disable this criterion. Required (no default); can be changed on restart.

%---- Beta-cooling for disk problems (DISK_BETA_COOL on; incompatible with COOLING)  
BetaCool_Beta      10      % cooling time in units of the local inverse orbital frequency: t_cool = beta/Omega  
BetaCool_Tirr      0       % irradiation floor temperature in K that gas relaxes toward (0 = no floor)

BetaCool_Beta/BetaCool_Tirr: If DISK_BETA_COOL is on, every active gas cell relaxes toward the irradiation temperature as \(u \rightarrow u_{\rm irr} + (u - u_{\rm irr})\,\exp(-\Delta t/t_{\rm cool})\) each timestep (exact exponential integration), with \(t_{\rm cool} = \beta/\Omega\) and \(\beta=\)BetaCool_Beta (dimensionless). \(\Omega\) is BOX_SHEARING_OMEGA_BOX_CENTER if BOX_SHEARING is set; otherwise it is computed from the local gravitational acceleration as \(\Omega=\sqrt{|{\bf a}|/|{\bf r}|}\), assuming a central mass at the coordinate origin. BetaCool_Tirr is in Kelvin, converted to a specific energy at startup with the default mean molecular weight; set =0 to cool toward zero (the global MinGasTemp floor still applies). DISK_BETA_COOL cannot be combined with COOLING. Both parameters are required (no defaults). (The values in the block above are illustrative, not code defaults.)

%---- Helmholtz equation-of-state table (EOS_HELMHOLTZ on, or NUCLEAR_NETWORK on)  
EosTable      ./helm_table.dat     % path to the Timmes & Swesty Helmholtz free-energy table

EosTable: If EOS_HELMHOLTZ is on (it is switched on automatically by NUCLEAR_NETWORK), the path (relative to the run directory, or absolute) of the Helmholtz free-energy table file read at startup. A copy is distributed with the code as src/eos/helmholtz/helm_table.dat (also available from the link in the EOS_HELMHOLTZ description). Required (no default); the run stops if the file cannot be read. Not used with EOS_ANEOS, which has its own table parameters.

%---- Nuclear Reaction Networks (NUCLEAR_NETWORK on)  
NuclearNetworkDataFile    ""              % path to network data directory (empty = built-in aprox13; required for SkyNet/Torch)  
NuclearBurningFloor_T     1.0e8           % minimum temperature [K] below which nuclear burning is not computed (default 1e8 K)  
NuclearBurningFloor_rho   0               % minimum density [code units] below which burning is not computed (default 0 = no floor)  
NuclearNSE_T_threshold    6.0e9           % temperature [K] above which NSE table used instead of integrating network (default 6e9 K)

NuclearNetworkDataFile: Path (as a string) to the directory containing reaction rate data files needed by the chosen nuclear network solver. For the built-in aprox13 network (NUCLEAR_NETWORK_SOLVER=0), this parameter is ignored and can be left as an empty string or omitted entirely – no external data files are needed. For external solvers (NUCLEAR_NETWORK_SOLVER=1 for SkyNet, =2 for Torch), this must be set to the directory containing the network’s data tables (e.g. nuclear partition functions, reaction rate coefficients). See the documentation of the respective library for what files are required and where they should live.

NuclearBurningFloor_T: Minimum temperature threshold (in Kelvin) below which nuclear burning reactions are not computed for a given particle. Particles with temperature below this value are assumed to be in a regime where the nuclear timescales are far longer than the hydro timestep and the energy release is negligible. The default value of \(10^8\) K is appropriate for most applications; lowering it will increase computational cost by integrating burning for cooler particles that contribute negligibly to the overall energetics. This parameter has no effect if NUCLEAR_NETWORK is not enabled.

NuclearBurningFloor_rho: Minimum density threshold (in code units) below which nuclear burning is not computed. Combined with NuclearBurningFloor_T, this allows you to exclude low-density regions (e.g. unbound ejecta) from the burning calculation for efficiency. The default value of 0 disables the density floor (i.e. burning is evaluated for any particle above the temperature floor regardless of density). For problems like white dwarf detonations where the outer, low-density layers are unimportant, setting this to a small positive value (e.g. the density of the initial surface layers) can significantly reduce compute time.

NuclearNSE_T_threshold: Temperature (in Kelvin) above which the code switches from integrating the full nuclear reaction network to a tabulated nuclear statistical equilibrium (NSE) lookup. At temperatures above this threshold, nuclear reactions proceed on timescales much shorter than the hydrodynamic timestep, so the composition can be assumed to be in instantaneous NSE, which is fast to evaluate from a pre-computed table. The default of \(6\times10^9\) K is a standard choice for thermonuclear burning applications; higher values will integrate the network to higher temperatures (more accurate but slower), while lower values will use NSE sooner (faster but less accurate for the approach to NSE). This parameter is only used when NUCLEAR_NETWORK_NSE_TABLE is enabled in Config.sh.

Radiation Transport

%-------------- Uniform background radiation field (RT_ISRF_BACKGROUND on)  
InterstellarRadiationFieldStrength   1.0    % ISRF normalization relative to the Solar-neighborhood (Draine) field (alt name ISRF; default 1)  
Redshift_RT_Background               0.0    % redshift of the CMB/UV background in non-cosmological runs (alt name z_background; default 0)  
%-------------- Infrared radiation (RT_INFRARED on)  
InitRadiationTemp                    20     % initial IR radiation and dust temperature in K (alt name InitRadTemp; default 20)  
%-------------- Idealized ionizing sources (RT_CHEM_PHOTOION on, without GALSF)  
IonizingLuminosityPerSolarMass_cgs   1.0e33 % ionizing luminosity per solar mass of star particle (erg/s/Msun)  
star_Teff                            1.0e5  % blackbody temperature (K) of the source spectrum

InterstellarRadiationFieldStrength (or ISRF): If RT_ISRF_BACKGROUND is on, this multiplies the assumed uniform Solar-neighborhood interstellar radiation field (Draine 1978/2011 values): the IR, optical/NIR, NUV and photo-electric (1.7 Habing) band energy densities used as the background in the explicit RT bands, the external field used in the equilibrium dust-temperature estimate, and the photo-electric \(G_{0}\) background in the cooling (attenuated by the local column). Default =1. Note the value given to the RT_ISRF_BACKGROUND Config flag itself only acts as a switch (a value \(\le 0\) un-defines it where it is auto-enabled); the normalization is set here.

Redshift_RT_Background (or z_background): If RT_ISRF_BACKGROUND is on, the redshift assumed for the CMB (temperature \(\propto 1+z\), energy density \(\propto (1+z)^{4}\)) and for the meta-galactic UV background tables in the cooling, in non-cosmological runs. In cosmological runs the actual simulation redshift is used and this is ignored. Default =0.

InitRadiationTemp (or InitRadTemp): If RT_INFRARED is on, the temperature (in K) used to initialize the IR radiation temperature, the dust temperature, and the IR radiation energy density (as a blackbody) of the gas on startup, and the IR background temperature. Values above 100 K are capped at 100 K where used. Default =20.

IonizingLuminosityPerSolarMass_cgs/star_Teff: Used only for idealized ionizing-RT problems: if RT_CHEM_PHOTOION is on and GALSF is not, every star particle (Type=4) is an ionizing source with luminosity equal to IonizingLuminosityPerSolarMass_cgs (erg/s per solar mass) times its mass, and the photo-ionization cross sections and photo-heating rates are averaged over a blackbody spectrum of temperature star_Teff (K). Both are required in that configuration (no defaults). (With GALSF the source luminosities come from the stellar-population model and \(T_{\rm eff}=4\times10^{4}\) K is hard-coded.) The values in the block above are illustrative only.

Star, Black Hole, and Galaxy Formation

%---- Star Formation parameters (GALSF on)  
CritPhysDensity     100.    %  critical physical density for star formation (cm^(-3))  
SfEffPerFreeFall    1.0     %  SFR/(Mgas/tfreefall) for gas which meets SF criteria  
%---- initialize metallicity+ages of gas+stars in simulation (for GALSF or metal-dependent COOLING)  
InitMetallicity              1e-5   % initial gas+stellar metallicity [for all pre-existing elements] (in solar)  
InitStellarAge               1      % initial mean age (in Gyr; for pre-existing stars in sim ICs)

CritPhysDensity: This sets the minimum physical density threshold (in particles per \({\rm cm^{3}}\)) for star formation. This is in addition to any other criteria which can be enabled at compile-time. In cosmological runs, we also always require that the physical density is >1000 times the mean cosmic density (i.e. at least well within a virialized structure), to ensure against completely spurious star formation at very high redshifts if the density threshold is set very low (if you wish to modify this, you must change All.CritOverDensity within the code).

SfEffPerFreeFall: This is the dimensionless SFR per free fall time (\({\rm SFR}/(M_{\rm gas} / t_{\rm freefall})\), where \(t_{\rm freefall} = \sqrt{3\pi /(32\,G\,\rho)}\)) assigned in the SF model to gas which meets -all- of the applied star formation criteria (e.g. is above the density threshold, and optionally also self-gravitating, molecular, etc). In sub-grid models such as the “effective equation of state” models, it is necessary to use this parameter to insert the Kennicutt-Schmidt (KS) relation “by hand”, setting it to a value ~0.005–0.02. For models with explicitly resolved feedback (e.g. the new FIRE models), the KS law is independent of the choice here, and we argue for the most physically accurate (and closest to individual SF simulations) choice of SfEffPerFreeFall=1.0 (together with the self-gravity criterion). For former GADGET users, know that this replaces the “MaxSfrTimescale” parameter, if you are using certain physics modules (e.g. FIRE) as opposed to the effective Springel and Hernquist EOS (in which case MaxSfrTimescale is still used). Note that if GALSF_EFFECTIVE_EQS is enabled, CritPhysDensity and SfEffPerFreeFall are not read at all (the code will warn they are being “ignored” – this is expected, not a bug): the effective-EOS model sets its own internal density threshold and star formation efficiency from MaxSfrTimescale, FactorSN, FactorEVP, TempSupernova, and TempClouds instead.

InitMetallicity (or Initial_Metallicity): Initial metallicity of all gas+stars in the simulation (in units of solar, abundance ratios are assumed solar). This is needed both for idealized galaxies, but even in cosmological integrations, various interpolation tables for stellar evolution or molecular cooling might not extrapolate well to truly primordial gas, so values ~1.0e–5 - 1e–3 are recommended if you are not sure about these (although often using a value of 0 is also safe). Obviously if you want to start from a pre-enriched state, or in non-cosmological simulations with finite metal abundances, this specifies the initial state as desired.

InitStellarAge (or Initial_StellarAge_inICs): Median initial age of all pre-existing star particles in the simulation initial conditions file (in Gyr), required so their feedback and mass loss can be calculated. The stars will individually be assigned random ages between zero and twice this value. In cosmological integrations, there are no initial stars so this is ignored.

%---- sub-grid (Springel+Hernquist/GADGET/AREPO) "effective equation of state"  
%------- star formation+feedback model (GALSF_EFFECTIVE_EQS on)  
MaxSfrTimescale     4.0         % code units (SF timescale at 2-phase threshold)  
TempSupernova       3.0e8       % in Kelvin (temp of hot gas in 2-phase model)  
TempClouds          1000.0      % in Kelvin (temp of cold gas in 2-phase model)  
FactorSN            0.1         % SNe coupling frac (frac of egy retained in hot)  
FactorEVP           3000.0      % controls Kennicutt normalization  
FactorForSofterEQS  1.0         % interpolate between 'stiff' and isothermal EOS  
%------- the sub-grid "decoupled winds" model (GALSF_SUBGRID_WINDS on)  
WindEfficiency          2.0     % mass-loading (Mdot_wind = SFR * WindEfficiency)  
WindEnergyFraction      0.06    % fraction of SNe energy in winds (sets velocity)  
WindFreeTravelMaxTime   0.1     % 'free-stream time' in units of t_Hubble(z)  
WindFreeTravelDensFac   0.1     % 'free-stream' until density < this * CritPhysDensity  
%------- alternative winds (set GALSF_SUBGRID_WIND_SCALING == 1 or 2)  
%------- (scaling with local dark matter properties, as Dave/Oppenheimer/Mannucci/Illustris)  
VariableWindVelFactor    1.0    % wind velocity relative to estimated halo v_escape  
VariableWindSpecMomentum 5000.  % wind momentum per unit stellar mass (code velocity units)

These factors all relate to the popular sub-grid model for star formation and feedback originally developed by Springel & Hernquist, MNRAS, 2003, 339, 289. We point the interested reader to that paper for the detailed description. This replaces explicit resolution of a multi-phase ISM (as well as any cooling below \(\sim 10^{4}\) K) with a stiff “effective equation of state” for gas above some modest density, motivated by a McKee & Ostriker-like analytic toy model for a two-phase ISM. This also includes sub-grid “decoupled” winds: essentially, for every unit mass stars formed, some proportional mass is forced out of the galaxy at a fixed velocity or at the escape velocity (depending on compile flags), with the “mass loading” set by hand. This is the model used in GADGET–2/3 and AREPO, and in many major simulation projects (“Illustris”, “OWLS”, “Eagle”, the Oppenheimer & Dave IGM studies, and more).

MaxSfrTimescale: This controls the normalization of the SFR. This specifically normalizes the SFR timescale at the threshold density for SF, so effectively determines the efficiency per free-fall time. Make sure you match this parameter to your choices for things like the critical density for star formation, in order to control the star formation rate at the level you actually intend to set it.

TempSupernova/TempClouds/FactorSN/FactorEVP: These control the sub-grid analytic model assumptions about the two-phase medium (temperature of “hot phase” gas, “cold clouds”, SNe coupling efficiency, and “evaporation” respectively): see the sub-grid model methods paper for details. Ultimately, these parameters control the exact “stiffness” of the sub-grid equation of state (its effective polytropic index) and the normalization/slope of the Kennicutt law (which is set explicitly by these parameters when the sub-grid models are used). The values above are plausible defaults, but users should experiment to see how this modifies the resulting EOS.

FactorForSofterEQS: Interpolation factor between the “full” or “stiff” Springel & Hernquist effective equation of state and a simple isothermal (\(10^{4}\) K) equation of state. Set=1 for the S+H effective EOS, =0 for the isothermal case; the interpolation is linear in pressure (at fixed density) in between (so if the ‘subgrid’ EOS has pressure \(P_{\rm subgrid}=100\), and the isothermal eos \(P_{\rm iso}=1\), then the interpolated EOS with \(q_{\rm eos}=\)FactorForSofterEQS\(=0.25\) will have \(P_{\rm effective}=q_{\rm eos}\,P_{\rm subgrid} + (1-q_{\rm eos})\,P_{\rm iso} = 25.75\).

WindEfficiency: Assumed wind mass-loading. The winds are made of particles/cells “kicked” stochastically, at a rate \(\dot{M}_{\rm wind} = \epsilon\,\dot{M}_{\ast}\) where \(\epsilon=\)WindEfficiency.

WindEnergyFraction: Fraction of SNe energy (time and stellar-mass integrated over the IMF and all ages of stellar populations) coupled in the winds. This sets the wind “kick” velocities. Test this to determine the launch velocity, the default assumes a Chabrier IMF with a standard \(10^{51}\) ergs per SNe.

WindFreeTravelMaxTime: Maximum assumed “free travel time” for the wind in units of the Hubble time at the current simulation redshift. If this is >0, the winds are “decoupled” (not allowed to interact via hydrodynamics) until they reach this time or WindFreeTravelLength. This is the standard in e.g. Illustris; for the assumption used in the OWLs/EAGLE simulations, set this =0 (no streaming) and boost the WindEnergyFraction instead.

WindFreeTravelDensFac: Winds will free-stream until they reach a density equal to this times the critical density at which SF is allowed. Set equal to zero to ignore (or some high value to always trigger it).

VariableWindVelFactor: If the config.sh parameter GALSF_SUBGRID_WIND_SCALING is included and set to 1 or 2, a different wind model is included which scales the wind properties with either halo mass or the local DM velocity dispersion. In that case, this parameter multiplies the wind outflow velocity relative to the “default” value estimated in that module, i.e. \(v_{\rm wind} = f\,v_{\rm default}\), where \(v_{\rm default}\) has appropriate units (e.g. the local DM velocity dispersion or virial velocity), so this parameter should be dimensionless.

VariableWindSpecMomentum: If the config.sh parameter GALSF_SUBGRID_WIND_SCALING is included and set to 1 or 2, a different wind model is included which scales the wind properties with either halo mass or the local DM velocity dispersion. In that case, this parameter determines the specific momentum of the wind for purposes of determining the mass-loading. In other words, this is the usual parameter \(p_{\ast}/m_{\ast}\) from the wind literature, so if you are forming stars at a rate \(\dot{M}_{\ast}\), the momentum injection rate going into winds is \(\dot{P}=(p_{\ast}/m_{\ast})\,\dot{M}_{\ast}\). The wind mass-loading follows from this and the wind velocity: \(\eta=\dot{M}_{\rm wind}/\dot{M}_{\ast}=(p_{\ast}/m_{\ast}) / v_{\rm wind}\). The parameter represents outflow momentum per unit stellar mass formed [momentum/mass], so it has the units of the code velocity.

%-------------- FIRE (PFH) explicit star formation & feedback model (FIRE on; all have 'alt name' alternative name options and defaults)  
%--- local coupling from LEBRON-first-stage of radiation-pressure photon momentum flux (GALSF_FB_FIRE_RT_LOCALRP)  
RP_Local_Momentum_Renormalization       1   % fraction of absorbed physical photon momentum to couple (alt name: 'WindMomentumLoading')  
%--- SneII Heating Model (GALSF_FB_MECHANICAL)  
SNe_Energy_Renormalization              1   % fraction of physical SNe energy to inject -before- cooling (alt name: 'SNeIIEnergyFrac')  
%--- HII region photo-heating model (GALSF_FB_FIRE_RT_HIIHEATING)  
HIIRegion_fLum_Coupled                  1   % fraction of ionizing photons allowed to see gas (alt name: 'HIIRegion_fLum_Coupled')  
%--- long-range radiation pressure acceleration (GALSF_FB_FIRE_RT_LONGRANGE)  
PhotonMomentum_Coupled_Fraction         1   % fraction of L to allow incident (alt name: 'PhotonMomentum_Coupled_Fraction')  
PhotonMomentum_fUV                      0   % minimum local-escape fraction in UV from incident SED, to propagate to long-range LEBRON component  
PhotonMomentum_fOPT                     0   % minimum local-escape fraction in optical from incident SED, to propagate to long-range LEBRON component  
%--- gas return/recycling  
StellarMassLoss_Rate_Renormalization    1   % fraction of gas mass returned, relative to ssp (alt name 'GasReturnFraction')  
StellarMassLoss_Energy_Renormalization  1   % fraction of returned gas energy+momentum, relative to ssp (alt name 'GasReturnEnergy')

These parameters control the stellar feedback models developed in the series of Hopkins, Quataert, and Murray papers, and summarized in the FIRE simulations (Hopkins et al. 2014). Unlike the sub-grid models described above, these simulations attempt to explicitly follow low-temperature gas cooling, a multi-phase ISM, and feedback from stars in the form of photo-ionization, SNe, stellar winds, radiation pressure, etc. We refer to the FIRE paper and the related papers studying the numerical aspects of these models (especially Hopkins et al., MNRAS, 2011, 417, 950 and Hopkins et al., MNRAS, 2012, 421, 3488) for details.

RP_Local_Momentum_Renormalization or WindMomentumLoading: Both names have identical functionality, just choose one (they both exist here and in other modules for historical compatibility). This parameter allows you to arbitrarily re-normalize the coupled momentum associated with the local coupling step (the first-stage of the LEBRON algorithm) of single-scattering and multiple-scattering from the IR of photons (i.e. radiation pressure). The values here and in most of the other feedback routines are normalized to the results of the IMF-averaged output of Starburst99 for a Kroupa IMF: so choosing 1.0 (recommended) follows exactly the time-dependent radiation and spectral shapes from the stellar evolution code, coupling only the actually absorbed photon momentum in the kernel around the star. Choosing a different value (0.1, say) doesn’t change the time or frequency-dependence, it just multiplies everything by that proportionality factor, if e.g. you wish to model a stellar population with a lower light-to-mass ratio, or assume some sub-grid model for un-resolved multiple scattering (which would require you set this -larger- than unity, which is not the default behavior) or cancellations (which would make this -smaller- than unity). Can also be set=0 (no coupling). (Default = 1)

SNe_Energy_Renormalization or SNeIIEnergyFrac: Fraction of the initial physical SNe energy (relative to what comes from the stellar evolution model) actually allowed to interact with gas. The code always allows cooling and already zeroes out any energy which would couple outside the cooling radius, so this should be the initial input energy from SNe explosions before any physical cooling, hence it should have the value =1 (the “efficiency” will be resolved), unless you intend to explicitly model e.g. a universe where SNe had systematically larger or smaller intrinsic energy compared to the default assumed (which is certainly possible within observational constraints, at the factor of 2 or so level). Can be set to 0 to disable SNe feedback. (Default = 1)

HIIRegion_fLum_Coupled: Multiplies the fraction of ionizing photons (relative to the full time-dependent stellar-evolution-code predicted values) allowed to interact with gas. Should be =1, again unless you specifically are trying to represent a non-standard IMF or other variant stellar population which would produce a different number of ionizing photons. Can be set to 0 to turn off photo-ionizing feedback. (Default = 1)

PhotonMomentum_Coupled_Fraction: Multiplies the actual physical, stellar-evolution predicted flux (relative to what is calculated to escape from stars, with their time-dependent spectra calculated from the SSP, then attenuated locally using the actual gas and dust in code following the LEBRON algorithm) which is then allowed to propagate via the gravity tree to have long-range radiative interactions. Should be =1 unless youre debugging. (Default = 1)

PhotonMomentum_fUV/PhotonMomentum_fOPT: Minimum fraction of the original UV/optical (respectively) portion of the stellar SED which is ‘forced’ to escape the vicinity of the star. This is used to e.g. represent an unresolved minimum escape fraction of ionizing radiation from local clouds, from ‘holes’ which are very small and therefore un-resolved in the simulations (in which case you would set these to some non-zero but likely small value). If you wish to use only the resolved attenuation in the simulations, these parameters should be set =0 (the default). (Default = 0, ignored)

StellarMassLoss_Rate_Renormalization or GasReturnFraction: Multiplies time-dependent stellar mass-loss return rates from O-star and AGB winds, relative to their stellar-evolution-calculated values. The corresponding energy, metals, momentum will be also multiplied by the same factor (the specific wind properties per unit mass and for a given age/metallicity SSP are not modified). Should be =1 (default value) unless again you are attempting to model a different IMF or other unusual stellar population. (Default = 1)

StellarMassLoss_Energy_Renormalization or GasReturnEnergy: Multiplies the returned gas specific thermal energy from O-star and AGB winds from stellar mass loss – effectively, this renormalizes the velocity of the stellar mass-loss products (multiplying them by the square root of this number), relative to that predicted by the standard stellar evolution models. Should be =1 for default behavior, but can be varied to represent the outputs of different stellar evolution models. (Default = 1)

Note that all of these FIRE parameters have sensible default values (the values in the parameter block above), which will be automatically used if the relevant code modules are turned on but the parameters are not specifically set by the user.

%-------------- ISM dust chemistry model (GALSF_ISMDUSTCHEM_MODEL on; all have defaults except AccretionTcutoffScaling)  
Initial_ISMDustChem_Depletion       0       % initial fraction of Si (and Fe) in dust, in gas below 1e5 K (alt name Initial_ISMDustChem_Dust_Depletion)  
Initial_ISMDustChem_SiltoCarbRatio  0       % initial silicate-to-carbonaceous dust mass ratio (alt name Initial_ISMDustChem_Silicate_to_Carbon_Dust_Ratio)  
SNeIIDustScaling                    1       % multiplies SNe II dust condensation efficiencies  
SNeIaDustScaling                    1       % multiplies SNe Ia dust condensation efficiency  
AGBDustScaling                      1       % multiplies AGB-wind dust yields  
DustAccretionScaling                1       % multiplies the rate of dust growth by gas-phase accretion  
ThermalSputteringScaling            1       % multiplies the thermal-sputtering destruction rate  
SNeGasClearedOfDustScaling          1       % multiplies the gas mass shocked by each SN in which dust is destroyed  
AccretionTcutoffScaling             1       % multiplies the 300 K gas-temperature ceiling for accretion (REQUIRED: no default)  
%--- grain-size evolution (GALSF_ISMDUSTCHEM_GRAINSIZEEVO on)  
UnitGrainNumber                     1       % unit of grain number used for the per-bin grain counts in ICs/snapshots (REQUIRED)  
UnitGrainLength_in_cm               1e-4    % grain length unit, written to snapshot headers (REQUIRED)  
ISMDustChem_Grain_Size_Min          1e-7    % minimum grain radius (cm)  
ISMDustChem_Grain_Size_Max          1e-4    % maximum grain radius (cm)  
SNeShatteringScaling                1       % multiplies the shattering efficiency in SNe shocks  
SNeSputteringScaling                1       % multiplies the sputtering efficiency in SNe shocks  
GrainShatteringScaling              1       % multiplies the grain-grain shattering rate in the ISM  
GrainCoagulationScaling             1       % multiplies the grain-grain coagulation rate in the ISM  
VCoagScaling                        1       % multiplies the coagulation threshold velocity  
CoagDensityEnhancementScaling       1       % multiplies the sub-resolution density enhancement (2000) used for coagulation in cold gas  
GrainVelocityScaling                1       % multiplies the turbulence-driven grain velocities  
PhotodestructionScaling             1       % multiplies the photodestruction rate (only if DUSTPHOTODESTRUCTION_TURNON is defined)

These parameters control the live ISM dust-evolution model enabled by GALSF_ISMDUSTCHEM_MODEL (Choban et al. 2022, arXiv:2112.09141, and subsequent papers; see the description of that flag above). Apart from the initial-condition parameters and the grain units, they are dimensionless multipliers of the default (calibrated) rates, so =1 gives the default model. Every one has a default except AccretionTcutoffScaling, UnitGrainNumber, and UnitGrainLength_in_cm, which must be set.

Initial_ISMDustChem_Depletion (or Initial_ISMDustChem_Dust_Depletion): Used only when initial dust abundances are set by the code (new runs, RestartFlag=0; also RestartFlag=2 if IO_DUST_NOT_IN_ICFILE is set). In gas cells colder than \(10^{5}\) K, this fraction of the Si (and, if iron dust is tracked, the Fe) is put into dust, with the other silicate elements depleted in silicate stoichiometric proportion. All initial dust is attributed to the SNe II source channel. With GALSF_ISMDUSTCHEM_GRAINSIZEEVO the initial dust gets an MRN (\(dn/da\propto a^{-3.5}\)) size distribution between ISMDustChem_Grain_Size_Min and ISMDustChem_Grain_Size_Max. Default =0 (no initial dust).

Initial_ISMDustChem_SiltoCarbRatio (or Initial_ISMDustChem_Silicate_to_Carbon_Dust_Ratio): With the above, the initial carbonaceous dust mass fraction is set to the silicate dust mass fraction divided by this ratio (capped at the total carbon abundance). Default =0, which is a division by zero; set it whenever Initial_ISMDustChem_Depletion > 0.

SNeIIDustScaling/SNeIaDustScaling: Multiply the dust condensation efficiencies of the key element of each dust species in SNe ejecta: 0.2 for SNe II (silicate, carbon, iron) and 0.005 for SNe Ia (iron), each capped at 1. Default =1.

AGBDustScaling: Multiplies the AGB-wind dust yields (tabulated cumulative yields as a function of age and metallicity). Default =1.

DustAccretionScaling: Multiplies the rate of dust growth by accretion of gas-phase metals onto grains in cold gas (it divides the reference growth timescale; with grain-size evolution, it multiplies the grain-radius growth rate). Default =1.

ThermalSputteringScaling: Multiplies the rate of dust destruction by thermal sputtering in hot gas (it divides the sputtering timescale; with grain-size evolution, it multiplies the erosion rate). Default =1.

SNeGasClearedOfDustScaling: Multiplies the mass of gas around each SN shocked to velocities fast enough to destroy dust (from the McKee 1989/Cioffi 1988 radiative-remnant scaling, or the Yamasawa et al. 2011 fit with grain-size evolution); the result is capped at the gas mass of the cell receiving the ejecta. Default =1.

AccretionTcutoffScaling: Multiplies the gas-temperature ceiling (\(300\) K) below which gas-phase accretion onto dust is allowed. With grain-size evolution, the same scaled ceiling also selects the cold gas that receives the sub-resolution density enhancement for coagulation. Set =1 for the default. Required: this parameter has no default, so it must appear in the parameterfile whenever GALSF_ISMDUSTCHEM_MODEL is on.

UnitGrainNumber/UnitGrainLength_in_cm: If GALSF_ISMDUSTCHEM_GRAINSIZEEVO is on, UnitGrainNumber is the unit in which the per-size-bin grain numbers are written to snapshots and read from ICs/snapshots (internally the counts are physical numbers; they are divided by this unit on output and multiplied by it on input). UnitGrainLength_in_cm is written to the snapshot header as the grain length unit, and is not otherwise used by the code. Both are required (no defaults).

ISMDustChem_Grain_Size_Min/ISMDustChem_Grain_Size_Max: Minimum and maximum grain radius (in cm) of the size grid: GALSF_ISMDUSTCHEM_GRAINSIZEEVO bins are spaced logarithmically between them. The minimum is also the smallest fragment size produced by shattering. Defaults \(10^{-7}\) and \(10^{-4}\) cm.

SNeShatteringScaling/SNeSputteringScaling: Multiply the species-dependent efficiencies with which grains are shattered and sputtered in gas shocked by SNe (grain-size evolution only). Default =1.

GrainShatteringScaling/GrainCoagulationScaling: Multiply the rates of grain-grain shattering and coagulation in the ISM from turbulence-driven grain collisions (grain-size evolution only). Default =1.

VCoagScaling: Multiplies the species- and size-dependent threshold velocity below which colliding grains coagulate. The scaled value is still capped at the shattering threshold. Default =1.

CoagDensityEnhancementScaling: In cold gas (below the accretion temperature ceiling), the density used for coagulation is enhanced by a factor rising as a power law from 1 at \(n_{\rm H}=0.1\,{\rm cm^{-3}}\) to a maximum of 2000 at \(n_{\rm H}\ge100\,{\rm cm^{-3}}\), to represent unresolved dense molecular gas. This parameter multiplies that maximum enhancement. Default =1.

GrainVelocityScaling: Multiplies the turbulence-driven grain velocities (Hirashita & Chen 2023 supersonic-turbulence scaling) that set grain-grain relative velocities and so the coagulation/shattering outcomes. Default =1.

PhotodestructionScaling: Multiplies the rate of photodestruction of small grains in recently photo-ionized gas. This term is compiled only if DUSTPHOTODESTRUCTION_TURNON is defined, which no Config template option sets; otherwise the parameter is read but has no effect. Default =1.

%-------------- sink particle accretion & formation (SINK_PARTICLES on)  
%--- formation/seeding  
SeedSinkMass            1.0e-7  % initial mass (on-the-fly or single galaxy)  
SeedReservoirMass       0.0        % initial mass in the alpha disk (SINK_ALPHADISK_ACCRETION)  
SeedSinkMinRedshift     2.0        % minimum redshift where new BH particles are seeded (lower-z ceases seeding)  
SeedSinkMassSigma       0.5        % lognormal standard deviation (in dex) in initial BH seed masses  
SeedSinkPerUnitMass     1.e-4      % stellar mass (code unit) per BH seed (SINK_SEED_FROM_LOCALGAS)  
%----- (specific options for on-the-fly friends-of-friends based BH seeding: FOF on)  
MinFoFMassForNewSeed    10.     % minimum mass of FOF group (stars or DM) to get seed, in code units  
TimeBetOnTheFlyFoF      1.01       % time (in code units, e.g. scale-factor) between on-the-fly FOF searches  
%--- accretion  
SinkAccretionFactor     1.0     % multiplier for mdot (relative to model)  
SinkEddingtonFactor     1.0     % fraction of eddington to cap (can be >1)  
SinkNgbFactor           3.0     % multiplier for kernel neighbors for BH  
SinkMaxAccretionRadius  0.02        % max radius for BH neighbor search/accretion (code units)  
SinkRadiativeEfficiency 0.1        % radiative efficiency (for accretion and feedback)  
%--- feedback  
SinkFeedbackFactor      1.0     % generic feedback strength multiplier  
Sink_Rad_MomentumFactor 1.0     % multiply radiation pressure (SINK_PHOTONMOMENTUM)  
Sink_accreted_fraction  0.5        % fraction of gas swallowed by sink (SINK_WIND options; alt name Sink_f_accretion)  
Sink_outflow_velocity   30000       % velocity (code units) of outflow from sink (SINK_WIND options; alt name Sink_v_outflow)  
Sink_outflow_temperature 1.0e4      % internal temperature (K) of BAL outflow (SINK_WIND_SPAWN; alt name Cell_Spawn_Temperature)  
Sink_outflow_particlemass 1.e-10     % mass of 'virtual wind particles' in code units (SINK_WIND_SPAWN; use Cell_Spawn_Mass_ratio if SINGLE_STAR or SINK_SCALE_SPAWNINGMASS_WITH_INITIALMASS is active)  
Sink_CosmicRay_Injection_Efficiency 0.1   % fraction of accretion energy into cosmic rays (SINK_COSMIC_RAYS; alt name Sink_CosmicRay_Injection_Efficiency)  
Sink_outflow_jetlaunchvelscaling 0.3      % jet launch speed in units of the Keplerian speed at the protostellar radius (SINGLE_STAR_FB_JETS; alt name Sink_f_launch_v)  
Sink_spawn_injection_radius 0.01         % radius (code units) for spawned-cell placement and B-field profile (SINK_WIND_SPAWN_SET_BFIELD_POLTOR)  
Sink_spawn_poloidal_B   1.e-6            % poloidal B amplitude of spawned cells, in units of UnitMagneticField_in_gauss (SINK_WIND_SPAWN_SET_BFIELD_POLTOR)  
Sink_spawn_toroidal_B   1.e-6            % toroidal B amplitude of spawned cells, in units of UnitMagneticField_in_gauss (SINK_WIND_SPAWN_SET_BFIELD_POLTOR)  
Sink_jet_precession_degree 10            % jet tilt angle (degrees) from the jet axis (SINK_WIND_SPAWN_SET_JET_PRECESSION)  
Sink_jet_precession_period 0.001         % jet precession period in Gyr (SINK_WIND_SPAWN_SET_JET_PRECESSION)

These parameters control the sub-grid models for super-massive black hole particles, enabled by the top-level compile-time switch SINK_PARTICLES. There are many different compile-time options for the seeding of BH particles, for their accretion physics, and for different feedback physics. These are described above (Config.sh section) However, they are mostly controlled by a common set of parameters (e.g. all accretion models use the same “SinkAccretionFactor” term).

SeedSinkMass: This gives the initial BH mass (in code units) when an on-the-fly seeding from the FoF method occurs. Likewise for any other on-the-fly seeding (for example, this determines the initial mass of the BHs in the on-the-fly local model, from SINK_SEED_FROM_LOCALGAS). In simulations which start with pre-existing BHs (from ICs), this should be set to the initial BH mass in code units (the particle mass will be read from the ICs, but GIZMO separately tracks the “sink” mass, and unless you construct special ICs that will not be read in). It’s easy to forget this! If you restart from a snapshot or restartfile, the BH masses will be read from the file.

SeedReservoirMass: Initial mass in the viscous alpha-disk (in code units), when SINK_ALPHADISK_ACCRETION is enabled. Usually zero, unless you want to initialize BHs rapidly accreting.

SeedSinkMinRedshift: Minimum redshift, below which seed BHs will not be spawned on-the-fly (regardless of the seed model in the simulations).

SeedSinkMassSigma: In the seed models which spawn BHs on the fly, this parameter allows them to have random masses drawn from a log-normal distribution. Then SeedSinkMass becomes the median/peak of the log-normal, and this specifies with width (in dex) of the distribution function. To fix the BH mass, simply set this to zero.

SeedSinkPerUnitMass: If SINK_SEED_FROM_LOCALGAS is enabled, this determines the seed probability (probability is dp=dmformed/this mass), set in code units

MinFoFMassForNewSeed: This sets the minimum mass of an on-the-fly FOF group which will get a BH seed (if a BH isn’t already present), when using the SINK_SEED_FROM_FOF flag[s]. The mass is in code units, and refers to either the total mass (if SINK_SEED_FROM_FOF=0, where total halo mass is used to identify groups) or stellar mass (if SINK_SEED_FROM_FOF=1, where BHs are seeded in stellar groups specifically).

TimeBetOnTheFlyFoF: This specifies the time delay (in code units) between searches for FOF groups for the BH seeding. Searches can happen more frequently, at snapshot points, for example, when the FOF finder would run anyways, but this will periodically force the search. If the run is cosmological, this is a scale-factor, so setting it to 1.1 will force a search every time the scale factor is a factor 1.1 larger (must be >1, in this case); in a non-cosmological run it is just the normal code time unit (like for snapshot spacing).

SinkAccretionFactor: Multiplies the accretion rate relative to the ‘default’ for whatever chosen accretion model is given by the compile-time choices. It can always be set =0 to prevent BH accretion.

SinkEddingtonFactor: Fraction of the Eddington limit at which to cap the maximum BH accretion rate. Set to some arbitrarily large value to effectively remove the Eddington limit.

SinkNgbFactor: The BH particle searches for neighbors to determine things like its accretion, BH-BH mergers, some feedback calculations. Depending on the fueling/feedback model (which may require estimating things like whether the gas is in a disk), you may want the number of neighbors to be several times larger than the ‘default’ kernel search. This multiplies DesNumNgb, specifically for the search around the BH particles.

SinkMaxAccretionRadius: This sets a hard maximum radius (in physical code units) out to which the BH particle will search. This is included to prevent cases where a poorly-resolved, gas-poor galaxy has a BH which then searches out to some very large distance until it finds gas (potentially leading to unphysical accretion or mergers from very large separations).

SinkRadiativeEfficiency: Radiative efficiency of the BH (defined in the standard manner, \(L =\) SinkRadiativeEfficiency \(*\,(dM/dt) * c^{2}\)). Used to calculate luminosities for feedback, eddington accretion rates, and correct the actual accreted mass.

SinkFeedbackFactor: Multiplier for the strength of all BH feedback mechanisms (whatever is enabled, the energy/momentum will be scaled by this factor relative to its ‘default’ which corresponds to values =1). You can set this =0 to turn off BH feedback.

Sink_FluxMomentumFactor or Sink_Rad_MomentumFactor: (Either name here produces the same functionality). If SINK_PHOTONMOMENTUM is on, this multiplies the long-range radiation-pressure force from sink luminosity, serving the same role as SinkFeedbackFactor but only for the radiation pressure terms. Currently this is separated just for de-bugging purposes. Default =1.

BAL_f_accretion/BAL_v_outflow (or Sink_f_accretion/Sink_v_outflow): These control the model for accretion disk or broad-absorption line winds from the BH (compiler flags beginning with SINK_WINDS). Sink_accreted_fraction (or Sink_f_accretion) controls the mass-loading of the winds: a fraction Sink_accreted_fraction of the mass in the inner accretion disk is assumed to be accreted, while (1-Sink_accreted_fraction) is blown out in the wind, with launch velocity Sink_outflow_velocity (or Sink_v_outflow) in km/s. See the Config.sh discussion, or the notes_blackholes file, for more details.

BAL_internal_temperature/BAL_wind_particle_mass or (Cell_Spawn_\Temperature/Cell_Spawn_Mass_ratio): These options are specific to the module SINK_WIND_SPAWN, and set the internal temperature (in K) and virtual particle mass (in code units) of the virtual “wind particles” spawned by the BH wind. Note that as described above, if the SINGLE_STAR modules are active or one sets the flag SINK_SCALE_SPAWNINGMASS_WITH_INITIALMASS, then the spawn mass is not fixed to a single absolute value but a value equal to Cell_Spawn_Mass_ratio times the initial formation mass of the sink particle doing the spawning.

Sink_CosmicRay_Injection_Efficiency: This determines the fraction of the accretion energy (not radiated energy) which is deposited into cosmic rays when SINK_COSMIC_RAYS is active

Sink_outflow_jetlaunchvelscaling (or Sink_f_launch_v): If SINGLE_STAR_FB_JETS is on, the launch speed of spawned jet cells is this factor times the Keplerian speed \(\sqrt{G\,M_{\rm sink}/R_{\ast}}\) at the protostellar radius \(R_{\ast}\) (from the protostellar evolution model if SINGLE_STAR_STARFORGE_PROTOSTELLAR_EVOLUTION is on, otherwise a fixed \(10\,R_{\odot}\)). For jets this replaces Sink_outflow_velocity. Default =0.3.

Sink_spawn_injection_radius/Sink_spawn_poloidal_B/Sink_spawn_toroidal_B: If SINK_WIND_SPAWN_SET_BFIELD_POLTOR is on, spawned cells are given a magnetic field with poloidal and toroidal components set in the frame of the launch axis. Each component has a Gaussian radial profile of scale Sink_spawn_injection_radius, normalized to its amplitude at that radius. The amplitudes Sink_spawn_poloidal_B and Sink_spawn_toroidal_B are in the same units as the initial-condition magnetic fields (UnitMagneticField_in_gauss). Sink_spawn_injection_radius is in physical code length units. It also caps the distance from the sink at which cells are spawned, and limits the timestep of the sink and spawned cells to Sink_spawn_injection_radius/Sink_outflow_velocity. All three are required (no defaults); the radius can be changed on restart.

Sink_jet_precession_degree/Sink_jet_precession_period: If SINK_WIND_SPAWN_SET_JET_PRECESSION is on, the launch axis of spawned jets is tilted by Sink_jet_precession_degree (degrees) and rotated about the \(z\)-axis with period Sink_jet_precession_period (in Gyr). The phase is \(2\pi\,t/P\) with \(t\) the simulation time, so this is not meaningful for cosmological runs. Both are required (no defaults).

Grackle Cooling Module

%-------------- Grackle UVB file (COOL_GRACKLE on)  
GrackleDataFile          CloudyData_UVB=HM2012.h5

GrackleDataFile: If the COOL_GRACKLE cooling modules are enabled, the user needs to specify the relevant data file for the pre-compiled UV background files used by Grackle. Additional flags may be needed for more advanced Grackle options.

CHIMES Chemistry Network

%-------------- CHIMES non-equilibrium chemistry (CHIMES on; none of these have defaults)  
Chimes_data_path        /path/to/chimes-data  % CHIMES data directory (download from the chimes-data repository)  
PhotoIonTable           FG20_cross_sections/z0.000_cross_sections.hdf5  % UV-field cross-section file, relative to Chimes_data_path (a directory if redshift_dependent_UVB_mode>0)  
EqAbundanceTable        DummyTable.hdf5       % equilibrium-abundance table, relative to the EqAbundancesTables sub-directory (used if Chemistry_eqm=1)  
Thermal_Evolution_On    1        % 0=evolve abundances at fixed temperature; 1=evolve temperature and abundances together  
Chemistry_eqm           0        % 0=time-dependent (non-equilibrium) chemistry; 1=equilibrium abundances from EqAbundanceTable  
redshift_dependent_UVB_mode 0    % 0=fixed UV field; 1=interpolate UVB cross sections+strength to current z; 2=also interpolate equilibrium tables  
reionisation_redshift   7.5      % before this redshift the redshift-dependent UVB is zero (used if redshift_dependent_UVB_mode>0) (example value; no default)  
InitIonState            1        % ionization state used to initialize abundances (CHIMES_INITIALISE_IN_EQM): 0=neutral, 1=singly ionized, ... (example value; no default)  
StaticMolCooling        0        % CO/H2O line cooling: 0=velocity-divergence (LVG) column; 1=thermal broadening only  
CellSelfShielding_On    1        % 0=optically thin; >0=self-shielding on (also needs CHIMES_SOBOLEV_SHIELDING)  
Shielding_length_factor 0.5      % multiplies the shielding length (CHIMES_SOBOLEV_SHIELDING)  
Grain_Temperature       10       % dust temperature (K) for H2 formation on grains and gas-grain energy exchange  
CR_rate                 1.8e-16  % cosmic-ray ionization rate of HI (s^-1); other species scaled from this  
max_mol_temperature     1.0e5    % molecules are excluded above this temperature (K)  
rad_field_norm_factor   1.0      % multiplies the UV field strength given by PhotoIonTable (the UVB only, if CHIMES_STELLAR_FLUXES)  
relativeTolerance       1.0e-4   % relative error tolerance of the thermo-chemistry integration  
absoluteTolerance       1.0e-10  % absolute abundance tolerance of the chemistry integration  
explicitTolerance       0.1      % take the explicit step if the relative change over the step is below this  
scale_metal_tolerances  0        % 1=scale absolute tolerances by each element abundance (recommended for cosmological runs)  
IncludeCarbon           1        % 0/1: exclude/include each element in the network (needs METALS + COOL_METAL_LINES_BY_SPECIES)  
IncludeNitrogen         1  
IncludeOxygen           1  
IncludeNeon             1  
IncludeMagnesium        1  
IncludeSilicon          1  
IncludeSulphur          1  
IncludeCalcium          1  
IncludeIron             1  
N_chimes_full_output_freq 10     % write the full abundance array every N-th snapshot (CHIMES_REDUCED_OUTPUT)  
chimes_debug            0        % 0=silent; 1=report CVODE errors/warnings; 2=also print the failing cell's gas variables  
Chimes_f_esc_ion        0.05     % fraction of stellar ionizing photons escaping the local HII region (CHIMES_STELLAR_FLUXES)  
Chimes_f_esc_G0         0.1      % fraction of stellar 6-13.6 eV photons escaping the local HII region (CHIMES_STELLAR_FLUXES)

These parameters are read only when CHIMES is on (Chimes_f_esc_ion and Chimes_f_esc_G0 only with CHIMES_STELLAR_FLUXES). None of them has a default, so every line above must be present in the parameterfile. The values shown are the typical values from the example block params_chimes.txt and the CHIMES mini user guide, both in the cooling/chimes folder. That mini guide (chimes_gizmo_doc.md, together with README.txt) describes each parameter in more detail and should be read before using CHIMES. Notes on parameters that the mini guide lists under different names, or does not list:

Driven Turbulence (Large-Eddy Simulations)

%-------------- Turbulent stirring parameters (TURB_DRIVING on)  
TurbDrive_ApproxRMSVturb            1.0     % driving strength: set to approximate rms turbulent velocity desired (code units)  
TurbDrive_MinWavelength             0.5     % minimum driving wavelength (code units): must be < MaxWavelength  
TurbDrive_MaxWavelength             1.0     % maximum driving wavelength (code units): must be <= BoxSize  
TurbDrive_SolenoidalFraction        1       % fraction in solenoidal modes (projection ~wt*curl + (1-wt)*div; default = 0.5)  
TurbDrive_CoherenceTime             1       % coherence/correlation time of driving (code units; default = MaxWavelength / RMSVturb)  
TurbDrive_DrivingSpectrum           2       % driving spectrum: 0=equal power all modes, 1=strongly-peaked between Max/Min, 2=Kolmogorov, 3=Burgers (default = 2)  
TurbDrive_RandomNumberSeed          42      % random integer seed for stochastic driver, for reproducibility (default = 42)  
TurbDrive_TimeBetweenTurbUpdates    0.01    % time interval (code units) for driving updates (default = 0.01*coherence time)  
TurbDrive_TimeBetTurbSpectrum       0.5     % time interval (code units) for outputting power spectra, if TURB_DRIVING_SPECTRUMGRID is on

These parameters control the optional module for stirred/driven turbulence (set by TURB_DRIVING), for large-eddy simulations (driven turbulent boxes and the like). They are based on a module written by Andreas Bauer for GADGET, but have been updated in a number of minor ways by PFH.

TurbDrive_ApproxRMSVturb or ST_energy: Both names produce identical functionality. This normalizes the characteristic rms acceleration summed over driving modes, as this value divided by the characteristic coherence time (defined below), such that for a linear, purely un-correlated Markov process with negligible pressure effects and no other forces and the continuum limit of infinite mode sampling over a narrow range in driving scales, the rms one-dimensional turbulent velocities averaged on scales larger than the driving scale and driving coherence time would be \(\delta v_{\rm drive}\) approximately equal to this value (set in code units). In other words, if you want to drive turbulence with an rms velocity \(=V\) in code units, set TurbDrive_ApproxRMSVturb=V. Now in practice, stochastic effects from the random number seed, finite driving and box scale effects, geometric effects and projection effects from the chosen solenoidal/compressive mix, nonlinear effects, pressure effects, etc, will all cause the actual rms turbulent velocities in the box to differ somewhat from the exact value here (differences at the factor of a couple level are common and expected, and differences can be larger for highly supersonic turbulence especially where non-linear dissipation effects can be very strong). So you will need to play with this value somewhat to get the “desired” magnitude of turbulence, if that is your goal (as opposed to fixing the rms value of the acceleration field strength, which is what this parameter actually does).

TurbDrive_MinWavelength/TurbDrive_MaxWavelength (or ST_Kmax/ST_Kmin): Minimum and maximum wavelengths (in code units) over which the driving operates. At least one wavelength must fit inside the box, so in finite box-size simulations, the maximum wavelength must be less than or equal to the linear box size. The minimum wavelength must be smaller than the maximum wavelength. The code will report which modes are used, so if these values are invalid, there will be no turbulent modes defined/reported when in stdout when the code begins. Note the legacy names map to wavelengths, not wavenumbers: ST_Kmin sets the maximum wavelength and ST_Kmax sets the minimum wavelength, and both are interpreted as lengths in code units (the code converts them to wavenumbers as \(2\pi/\lambda\)). Old parameterfiles that gave ST_Kmin/ST_Kmax as wavenumbers must be converted. These parameters (and TurbDrive_ApproxRMSVturb) have no defaults and must be set.

TurbDrive_SolenoidalFraction or ST_SolWeight: Both names produce identical functionality (regardless of which module version you use, but only one can be defined or read in, so pick one convention). This sets the fractional weight \(w\) of solenoidal (aka incompressible, aka divergence-free) modes, with all the driving modes projected to a curl-like (solenoidal) and divergence-like (compressive) component. The projection is done such that the power of the driving scales as the weighted sum: \(= w\,\hat{\bf a}_{\rm solenoidal} + (1-w)\,\hat{\bf a}_{\rm compressive}\) (applied in Fourier space before transforming back to real space, where this operation is purely a linear projection operation of the different Fourier modes). So \(w=0\) corresponds to purely compressive driving (e.g. shocks, outflows, explosions, etc.), \(w=1\) purely solenoidal (e.g. stirring pure vorticity). For sufficiently sub-sonic (aka weakly-compressible) turbulence, \(w\approx1\) is usually appropriate, for highly super-sonic (weakly pressurized) turbulence empirical simulation results argue that the natural mix which arises is \(w\sim0.5\). If this is not specified, the code will default to \(w=0.5\).

TurbDrive_CoherenceTime or ST_decay: Both names produce identical functionality. Decay time (for exponential de-correlation) of the driving-mode phase correlations (at the largest driving scale) – i.e. the (Markov or Fokker-Planck-type) coherence timescale of the turbulent accelerations of the driving modes. Physically this should usually be something like the eddy turnover time on the maximum scale. This is in code units. If not specified, the code will default to assume this is given by the expected eddy turnover time at the largest driving scale, i.e. \(\lambda_{\rm drive} / \delta v_{\rm drive}\), where \(\lambda_{\rm drive} = 2\pi / k_{\rm min}\) (in terms of the minimum driving wavenumber \(k_{\rm min}\)) is the largest driving wavelength and \(\delta v_{\rm drive} = (\dot{e}_{\rm diss}\,t_{\rm coherence})^{1/2}\) (in terms of the coherence time itself and energy dissipation rate \(\dot{e}_{\rm diss}\)) is the expected rms velocity being driven at this scale.

TurbDrive_DrivingSpectrum or ST_SpectForm: Both names produce identical functionality. Sets the power spectral shape of the driving (the distribution of power specifically over the driving range of wavelengths). For a value =0 , the power is constant (\(d^{N}\bar{k}\,E_{\bf \bar{k}}\sim\) constant, where \(E_{\bf \bar{k}}\) is the power per \(N\)-dimensional mode, i.e. the energy integral is over \(\Delta E \sim E_{\bf \bar{k}} d^{N}{\bf \bar{k}}\), where \(N\) is the number of spatial dimensions). For =1 the driving is peaked sharply around the mean of the minimum and maximum driving wavelengths, with a width contained between the two – use this to approximate \(\delta\)-function like driving. For =2 the driving has a Kolmogorov spectrum (\(E_{k}\propto k^{-5/3}\) in terms of the isotropic \(\Delta E \sim E_{k} dk\), or \(E_{\bf \bar{k}}\propto |{\bf \bar{k}}|^{-N-2/3}\)). For =3 it has a Burgers spectrum (\(E_{k}\propto k^{-2}\) or \(E_{\bf \bar{k}}\propto |{\bf \bar{k}}|^{-N-1}\)). If this is not set, the code will default to assume the Kolmogorov-type form (=2).

TurbDrive_RandomNumberSeed or ST_Seed: Both names produce identical functionality. Random integer seed for the random number generator for the modes. This is useful so that different simulations can reproduce the same driving field, even though the field is based on a random number generator. If not specified, will be set =42.

TurbDrive_TimeBetweenTurbUpdates or ST_DtFreq: Both names produce identical functionality. Time interval between which the code updates the stochastic variables that control the turbulent driving fields. Should be much less than the specified coherence/correlation time, or unphysical values will result, but once this is true, the results converge rapidly with respect to this choice. If not specified (or explicitly set to a negative number), the code will default to updating the fields every 1% of the coherence timescale.

TurbDrive_TimeBetTurbSpectrum or TimeBetTurbSpectrum: Both names produce identical functionality. If TURB_DRIVING_SPECTRUMGRID is set, this specifies the time interval (in code units) at which the power spectra are output on-the-fly as the simulation runs.

Reminder, for a truly ‘isothermal’ test with sound speed unity, set EOS_ENFORCE_ADIABAT=1 and EOS_GAMMA=1.001 (or some other extremely close-to-one number), and the temperatures will be re-set as if EOS_GAMMA=1 but various unphysical divergences will be avoided (see notes for the TURB_DRIVING module above).

Non-Standard Dark Matter, Dark Energy, Gravity, or Expansion

%-------------- Parameters for non-standard or time-dependent Gravity/Dark Energy/Expansion (GR_TABULATED_COSMOLOGY on)  
DarkEnergyConstantW       -1        % time-independent DE parameter w, used only if no table  
TabulatedCosmologyFile    CosmoTbl  % table with cosmological parameters

%-------------- Parameters for self-interacting dark matter (DM_SIDM on)  
DM_InteractionCrossSection 1        % cross-section per unit mass in cm^2/g (normalized at 1 km/s, if vel-dependent)  
DM_InteractionVelocityScale 0       % velocity [code units]: if>0, cross-section scales ~1/(1+(v/v0)^4) above this "v0"  
DM_DissipationFactor 0              % 0=elastic, 1=pure dissipative (fractional dissipation of kinetic energy in event)  
DM_KickPerCollision  0              % velocity 'kick' [code units] per collision (this^2=specific energy/mass released)

%-------------- Parameters for scalar-field dark matter (DM_FUZZY on)  
FuzzyDM_Mass_in_eV 1e-22            % boson mass in eV for SFDM/Fuzzy DM module

%-------------- Parameters for scalar-field screened gravity (DM_SCALARFIELD_SCREENING on)  
ScalarBeta              0.1         % strength of the scalar-field force relative to gravity (dimensionless)  
ScalarScreeningLength   1.0         % screening (Yukawa) length of the scalar field, in comoving code length units

%-------------- Parameters for DM annihilation/decay heating (DM_HEATING on)  
DM_AnnihilationSigmaV_over_mChi 0   % <sigma v>/m_chi in cm^3 s^-1 g^-1 (self-conjugate convention, no extra 1/2)  
DM_AnnihilationHeatingFraction  0   % f_h^ann in [0,1] -- fraction of rest-mass-energy thermalized locally  
DM_DecayRate                    0   % Gamma in s^-1  
DM_DecayHeatingFraction         0   % f_h^dec in [0,1] -- fraction of m_chi c^2 thermalized locally per decay

%-------------- Parameters for the Continuum Vlasov (CBE) integrator (CBE_INTEGRATOR on)  
CBEMassEffFloor          0.1   % floor fraction (of cell mass) below which a basis is exempted from the per-basis mass-flux timestep criterion, so near-empty placeholder/free-slot bases cannot force a pathologically small step  
CBECollisionCrossSection 0     % sigma/mu_p [cross-section per unit mass, code units], for the intra-cell CBE collision operator (CBE_INTEGRATOR_COLLISIONS on); 0 = disabled (no-op)

DarkEnergyConstantW: If GR_TABULATED_COSMOLOGY is on, but none of the flags to actually read tabulated data are set (GR_TABULATED_COSMOLOGY_W, GR_TABULATED_COSMOLOGY_H, GR_TABULATED_COSMOLOGY_G), simple setting this parameter allows you to use an arbitrary constant (time-independent) value of the Dark Energy equation-of-state parameter “w”. If the table is read, this will be ignored.

TabulatedCosmologyFile: If the tabulated data is read, this specifies the name (file-path) of the file which contains the tabulated time-dependent cosmological parameters. The code will assume the file is standard ASCII text, with the relevant data in 5 columns separated by spaces. The format should be:

DM_InteractionCrossSection: If DM_SIDM is on, this sets the dark matter (DM) self-interaction cross section, in cgs units. If the cross-sections are velocity-dependent, then this is the normalization defined as \(\Delta v\rightarrow0\) (i.e. at zero velocity separation, so that the interactions are bounded).

DM_InteractionVelocityScale: If DM_SIDM is on, this sets the dark matter (DM) self-interaction cross section dependence on velocity. Specifically, if this is set to be \(>0\), then the cross section is assumed to scale as \(\sigma(v) = \sigma_{0}/[1 + (v/v_{0})^{4}]\) where \(\sigma_{0}\) is the value of DM_InteractionCrossSection and \(v_{0}\) is the value of DM_InteractionVelocityScale – i.e. this is given as a velocity in code units. Note by default we assume Rutherford-type scattering at high velocities (\(\propto v^{-4}\)): if desired, it is trivial to change this by editing the one line where this scaling appears in the function prob_of_interaction in sidm_core.c. If the parameter is set \(<=0\), then it will have no effect (i.e. the scattering is velocity–independent).

DM_DissipationFactor: If DM_SIDM is on, this sets the fractional degree of dissipation per interaction event. This interpolates between purely elastic scattering when equal to 0, to purely momentum-conserving scattering (both particles move at the center-of-momentum, i.e. all kinetic energy in the center-of-mass frame is entirely dissipated, so this is the most dissipative possible) when equal to 1. For intermediate values, this multiplies the post-collision recoil velocity in the center-of-mass frame, so the center-of-mass frame kinetic energy is decreased by a factor of this squared.

DM_KickPerCollision: If DM_SIDM is on, this sets the velocity of ‘kicks’ given to the DM in scattering events (beyond what would occur from pure elastic scattering). More specifically, this value (in code units) squared defines the specific energy (energy per unit mass) added to the kinetic energy of the scattering DM particles in the rest frame of the scattering event.

FuzzyDM_Mass_in_eV: If DM_FUZZY is on, this sets the mass of the DM boson (in eV). This effectively entirely determines the DM dynamics, in models where the DM is a scalar field, Bose-Einstein condensate, quantum wave, etc. Note that it is critical in these modules to actually resolve the DM de Broglie wavelength, so make sure you dont use too large a value of this parameter for your mass resolution.

ScalarBeta/ScalarScreeningLength: If DM_SCALARFIELD_SCREENING is on, DM particles feel an additional Yukawa-type scalar-field force on top of gravity: between two DM particles the extra force is ScalarBeta times the Newtonian force times \((1 + r/\lambda)\,e^{-r/\lambda}\), with \(\lambda=\)ScalarScreeningLength in (comoving) code length units. This is applied in both the tree and, with PMGRID, the PM long-range force. ScalarBeta is dimensionless. Both are required (no defaults). The values in the block above are illustrative.

DM_AnnihilationSigmaV_over_mChi: If DM_HEATING is on, this is the velocity-averaged annihilation cross section divided by the DM particle mass, \(\langle\sigma v\rangle/m_\chi\), supplied in cgs (cm$3$ s${–1}$ g${–1}$). The single combined parameter avoids two-knob coupling; setting it to zero disables the annihilation channel entirely. Convention: self-conjugate DM with no extra factor of \(1/2\) in the annihilation rate (volumetric rate \(\tfrac{1}{2}n_\chi^2\langle\sigma v\rangle\) combined with energy \(2 m_\chi c^2\) per event gives \(\rho_{\rm DM}^2 (\langle\sigma v\rangle/m_\chi) c^2\)). Dirac/non-self-conjugate DM users fold the appropriate factor of two into the supplied scalar. v-dependence (e.g. Sommerfeld) is also folded by the user into this scalar — there is no in-code velocity-callback in this version.

DM_AnnihilationHeatingFraction: If DM_HEATING is on, the fraction \(f_h^{\rm ann} \in [0,1]\) of the released rest-mass-equivalent energy that thermalizes locally in the gas (the remainder escapes as e.g. neutrinos or radiation that streams freely on the local scale). Hard-aborts at startup if outside \([0,1]\). Set to zero (along with DM_AnnihilationSigmaV_over_mChi) to disable the annihilation channel.

DM_DecayRate: If DM_HEATING is on, the DM particle decay rate \(\Gamma\) in cgs (s${–1}$). A sterile-neutrino X-ray-line scenario corresponds to \(\Gamma\) determined by \((m_s, \sin^2 2\theta)\). Setting this to zero disables the decay channel. The implementation neglects DM mass loss from decay (\(\Gamma t \ll 1\) assumed for any realistic decaying-DM run).

DM_DecayHeatingFraction: If DM_HEATING is on, the fraction \(f_h^{\rm dec} \in [0,1]\) of \(m_\chi c^2\) per decay that thermalizes locally in the gas. For a 2-body decay \(\chi \to \gamma + \nu\) where only the photon couples to gas, this is approximately \(1/2\) (the photon carries half the rest-mass energy). Hard-aborts if outside \([0,1]\).

CBEMassEffFloor: If CBE_INTEGRATOR is on, the floor fraction (of the total cell mass \(m_{a}\)) below which a basis’ mass \(m_{a,\alpha}\) is exempted from the per-basis mass-flux timestep criterion (see the method paper, \S~“Timestep Constraints”). Without this floor, a near-empty basis (e.g. an empty free slot, or one freshly reset by the realizability-repair operator) receiving a cell-scale flux would force a pathologically small timestep; the floor regularizes the effective mass used in that criterion to \(m_{a,\alpha}^{\rm eff} = \max(m_{a,\alpha},\,{\tt CBEMassEffFloor}\times m_{a})\). Default 0.1 if not set in the parameterfile.

CBECollisionCrossSection: If CBE_INTEGRATOR_COLLISIONS is on, this sets the intra-cell collision cross-section per unit (physical) particle mass, \(\sigma/\mu_{p}\), in code units. This is the actual physical control for the collision operator: the flag alone only compiles in the machinery, and the operator is a no-op (zero collision rate) unless this parameter is set >0. Setting it to zero (the default) leaves the module compiled but inert.

1.00 -1.00 1.00 1.00 1.00
0.90 -1.05 0.90 1.01 1.00

Here Column (1) is scale factor (\(a=1/(1+z)\)). Column (2) is the Dark Energy equation-of-state \(w(z)\), used if GR_TABULATED_COSMOLOGY_W is enabled. Column (3) is the Hubble function \(E(z)\), e.g. for concordance \(\Lambda\)-CDM is just \(E(z) = \sqrt{\Omega_{m}\,a^{-3}+\Omega_{K}\,a^{-2}+\Omega_{\Lambda}}\) – note the normalization is still set by \(H_{0}\) set as usual in the cosmological units of the parameterfile. This is used if GR_TABULATED_COSMOLOGY_H is enabled. If you want to use this with “standard” dark energy, just set the \(w(z)\) values to \(-1\) at all times. Columns (4)-(5) are used only if GR_TABULATED_COSMOLOGY_G is enabled. (4) Re-normalizes the gravitational contant \(G\) at each time – the default is the value set in the usual fashion in either the parameterfile above, or by the code, and then it will be multiplied by this number. Column (5) multiples the Hubble function by an appropruate multiplicative correction factor at each time. If you have pre-tabulated the correct \(H(z)\) yourself already, just write this into column (3) and set column (5) to unity at all times.

Solid Body and Impact Simulations

%-------------- Parameters for custom Tillotson equation-of-state (EOS_TILLOTSON on)
%--- In ICs, set "CompositionType": 0=custom,1=granite,2=basalt,3=iron,4=ice,5=olivine/dunite,6=water;
%---   their EOS parameters will be set accordingly. If CompositionType=0, the custom parameters below
%---   are used, matched to the definitions in Table A1 of Reinhardt+Stadel 2017,MNRAS,467,4252 (below is iron)
Tillotson_EOS_params_a          0.5     % a parameter [dimensionless]
Tillotson_EOS_params_b          1.5     % b parameter [dimensionless]
Tillotson_EOS_params_u_0        9.5e10  % u_0 parameter in [erg/g]
Tillotson_EOS_params_rho_0      7.86    % rho_0 parameter in [g/cm^3]
Tillotson_EOS_params_A          1.28e12 % A parameter in [erg/cm^3]
Tillotson_EOS_params_B          1.05e12 % B parameter in [erg/cm^3]
Tillotson_EOS_params_u_s        1.42e10 % u_s parameter in [erg/g]
Tillotson_EOS_params_u_s_prime  8.45e10 % u_s^prime parameter in [erg/g]
Tillotson_EOS_params_alpha      5.0     % alpha parameter [dimensionless]
Tillotson_EOS_params_beta       5.0     % beta parameter [dimensionless]
Tillotson_EOS_params_mu         7.75e11 % elastic shear modulus in [erg/cm^3]   (used if EOS_ELASTIC is on)
Tillotson_EOS_params_Y0         8.5e10  % hugoniot elastic limit in [erg/cm^3]  (used if EOS_ELASTIC is on)

These parameters are used if the EOS_TILLOTSON and/or EOS_ELASTIC flags are used to activate the equations-of-state and elastic dynamics for different solid+vapor bodies (for e.g. impact simulations or solid-body interactions). When this flag is active, all particles of Type=0 have an assigned material composition specified by the parameter “CompositionType”. This will invoke pre-tabulated equations of state for granite, basalt, iron, ice, olivine/dunite, or water (as described above), but if CompositionType=0, a custom EOS will be used, specified by the parameters in the parameterfile (if no particles have CompositionType=0, then these parameters will be ignored). These are specified above, in cgs units, following the notation convention in Reinhardt+Stadel 2017,MNRAS,467,4252. The example given is for iron, but any additional species can be added this way. Note that if EOS_ELASTIC is used without EOS_TILLOTSON, the shear modulus (mu) and elastic limit (Y0) from the custom CompositionType=0 slot are applied as universal constants to all particles.

If EOS_ANEOS is active instead of (or in addition to) EOS_TILLOTSON, the following runtime parameters specify the ANEOS/SESAME tabulated equation of state:

%-------------- Parameters for ANEOS/SESAME tabulated equation-of-state (EOS_ANEOS on)
AneosNumMaterials               2                       % number of ANEOS material tables to load (max 7)
AneosTable0                     forsterite.sesame       % file path for material 0 (CompositionType=0)
AneosTable1                     iron.sesame             % file path for material 1 (CompositionType=1)

AneosNumMaterials sets how many SESAME table files to load at startup. AneosTable0, AneosTable1, etc. specify the file path for each material’s table. Only the first AneosNumMaterials table paths are required; unused slots are ignored. Each particle’s CompositionType field (set in the initial conditions) selects which table to use (0-indexed). The tables should be in ASCII SESAME format with a header line containing the material ID, number of density points, number of temperature points, and number of columns, followed by the density grid, temperature grid, and 2D data rows. Tables in the Stewart group SWIFT format can be converted with the convert_stewart_table.py utility provided in the test suite.

If PLANET_HEATING is active (this automatically enables EOS_TILLOTSON), the following parameters set a uniform internal heating rate of all active Type=0 cells:

%-------------- Radiogenic + accretional heating of solid bodies (PLANET_HEATING on)
PlanetHeating_RadQ0_cgs     1.0e-7      % initial radiogenic specific heating rate [erg/g/s]
PlanetHeating_RadTau_cgs    3.3e13      % e-folding time of the radiogenic heating [s]; <=0 disables the radiogenic term
PlanetHeating_AccQ0_cgs     0           % constant background (accretional) specific heating rate [erg/g/s]; 0 = off

Each timestep the specific internal energy of every active Type=0 cell is increased by \((Q_{\rm rad}\,e^{-t/\tau_{\rm rad}} + Q_{\rm acc})\,\Delta t\), with \(t\) the simulation time. Here \(Q_{\rm rad}=\)PlanetHeating_RadQ0_cgs, \(\tau_{\rm rad}=\)PlanetHeating_RadTau_cgs (radiogenic heating from short-lived isotopes such as \(^{26}$Al and $^{60}$Fe; set $\tau\) large for near-constant heating), and \(Q_{\rm acc}=\)PlanetHeating_AccQ0_cgs (unresolved accretional heating; resolved impact heating is already in the hydro solver). All three are given in cgs and converted to code units at startup, and all three are required (no defaults). The values above are illustrative (the \(\tau\) shown is the ${26}$Al e-folding time, ~1.03 Myr).

Developer-Mode Parameters

All numerical methods include purely numerical parameters (for example, the Courant Factor), that control things like the error tolerance of the method. There is never a single value of these parameters that works best for all possible problems. However, they should only be modified if you really know what you are doing, or else the results can be disastrous (the code might look fine, but simply give you the wrong answer). GIZMO therefore defaults to hard-coding these to ‘compromise’ values that work well on a diverse array of test problems (see the methods paper). However, if you enable DEVELOPER_MODE in Config.sh, you can manually set these to whatever values you desire. BE CAREFUL!!!!

%--- Developer-Mode Parameters (usually hard-coded, but set manually if DEVELOPER_MODE is on) --------  
ErrTolTheta                  0.7    % 0.7=standard  
ErrTolForceAcc               0.0025 % 0.0025=standard  
ErrTolIntAccuracy        0.010  % <0.02  
CourantFac               0.1    % <0.20  
MaxRMSDisplacementFac    0.125  % <0.25  
MaxNumNgbDeviation      0.05    % <<DesNumNgb (values<1 are fine)  
AGS_MaxNumNgbDeviation  0.10  % tolerance in Nngb (make larger than gas)  
%---- Magneto-Hydrodynamics Developer-Mode Parameters (MAGNETIC on)  
%--- Dedner 2002 div-cleaning parameters  
DivBcleaningParabolicSigma   1.0 % (3D~0.7-1.0,2D~0.2-0.3)  
DivBcleaningHyperbolicSigma  1.0 % (~1)

ErrTolTheta: This is the accuracy criterion (the opening angle theta) of the tree algorithm if the non-standard Barnes & Hut (BH) opening criterion is used. If TypeOfOpeningCriterion=1 (this is assumed as the default in the code), theta and the BH criterion are only used for the first force computation. These first forces are only used as input for a recomputation of the forces with the relative cell opening criterion, which is employed thereafter for the rest of the force computations (this is more accurate and adaptive to irregular simulation geometries).

ErrTolForceAcc: This controls the accuracy of the relative cell-opening criterion (if enabled), where a cell is opened if \(M\,L^{2} > \alpha\,|a_{\rm old}|\,r^{4}\). Here, the dimensionless \(\alpha\) is set by ErrTolForceAcc, \(M\) is the mass inside a node, \(L\) is its cell side-length, and \(a_{\rm old}\) is the magnitude of the total acceleration of the particle. The latter is usually estimated based on the force on the particle in the previous timestep. Note that independent of this relative criterion, the code will always open nodes if the point of reference lies within the geometric boundaries of the cubical cell (enlarged by an additional 10% on each side). This protects against the rare occurrence of very large force errors, which would otherwise be possible. Values <0.005 are recommended, since this essentially sets the maximum fractional error for a single force calculation from a far-away node owing to higher-order moments (it will usually average-down to much smaller errors if a larger value is used, but this can generate significant noise).

ErrTolIntAccuracy: This dimensionless parameter controls the accuracy of the timestep criterion. GIZMO uses several timestep criteria, to prevent errors and numerical instabilities it always chooses the most restrictive of all possible timestep restrictions. This parameter corresponds to the ‘standard’ criterion in N-body simulations (see Powers et al., 2003, MNRAS, 338, 14), given by \(\Delta t=\sqrt{2\,\eta\,\epsilon/|a|}\), where \(\eta=\) ErrTolIntAccuracy, \(\epsilon\) is the gravitational softening length, and \(|a|\) is the magnitude of the total acceleration (from all sources) acting on a particle. For hydro particles, \(\epsilon\) is replaced by the minimum of either \(\epsilon\) or the kernel length/cell size. Testing suggests a maximum of 0.02 for test problems, but if orbits need to be followed for more dynamical times, lower values will be needed. Other timestep restrictions in GIZMO include the Courant criterion, a divergence criterion to prevent more than ~10% changes in the kernel length/cell size and gas density in a single timestep, an absolute maximum/minimum set by MaxSizeTimestep/MinSizeTimestep, the ‘displacement’ criterion specified by MaxRMSDisplacementFac, and physics-specific restrictions (for example, limiters to ensure small changes in properties owing to diffusion, conduction, star formation, dark matter self-interactions, stellar mass loss, restrictions to <1 SNe per timestep and small BH growth, cooling time restrictions, etc.). It is not uncommon for a particle to check ~10 different timestep restrictions.

CourantFac: This sets the value of the Courant parameter used in the determination of the hydrodynamical timestep (based on the maximum signal velocity and the gas kernel length or cell size); the convention for this is slightly different than in GADGET in order to allow more flexibility for different kernel choices and hydro methods. But a value of 0.2 here is equivalent to a value of 0.5 in cartesian grid codes, so that’s the “standard” maximum recommended value.

MaxRMSDisplacementFac: This defines an additional timestep limiter for the PM-step when the TreePM method is used. In this case, the code computes the 3D velocity dispersion (\(\sigma\)) for all particles in the simulation, and restricts the maximum simulation timestep (and hence the PM timestep) such that (\(\sigma\,\Delta t\)) is less than MaxRMSDisplacementFac times the mean particle separation “d”, or less than MaxRMSDisplacementFac times the scale r_s of the short-range/long-range force split, whichever is smaller. Values <0.25 are always recommended.

MaxNumNgbDeviation: The allowed variation of the effective number of neighbours around the target value DesNumNgb. Because of the continuous nature of this, non-integer values are fine. Should be much less than DesNumNgb for good conservation.

AGS_MaxNumNgbDeviation: This functions identically to MaxNumNgbDeviation, for adaptive gravitational softening kernels. AGS_MaxNumNgbDeviation can in some cases be set to values significantly larger than MaxNumNgbDeviation (if there is no hydrodynamic-like hyperbolic equation being solved for the relevant softened particles), but this should be done with care, and the value should still be small relative to AGS_DesNumNgb.

DivBcleaningParabolicSigma/DivBcleaningHyperbolicSigma: Dimensionless parameters which control the normalization of the parabolic and hyperbolic components of the divergence-cleaning scheme. The “default” values =1 will use the same values recommended by Tricco+Price (for SPH) or Gaburov & Nitadori 2010 (for non-SPH); see those papers for details.

When running the code with an SPH hydro solver, some additional parameters are required to control the artificial dissipation terms needed to stabilize all SPH methods. These are usually set to recommended values, unless DEVELOPER_MODE is enabled, then they can be set manually

%---------- SPH-Specific Developer-Mode Parameters ---------------------------------  
%---- Artificial viscosity  
ArtBulkViscConst    1.0     % multiplies 'standard' AV (use 1.0)  
%---- P&M artificial conductivity (if present); normalized to Alpha_Visc:  
ArtCondConstant     0.50    % multiplies 'standard' (use 0.25-0.5)  
%---- Cullen & Dehnen viscosity suppression  
ViscosityAMin       0.05    % minimum viscosity away from shocks (>0.025)  
ViscosityAMax       2.00    % maximum viscosity in shocks (>1)  
%---- Artificial resistivity (MAGNETIC on)  
ArtificialResistivityMax 1. % maximum alpha_B (~1-2) for art. res. (like art. visc)

ArtBulkViscCons: This multiplies the value of the dimensionless artificial viscosity parameter \(\alpha_{\rm visc}\) (see GADGET–2 code paper for details). A value =1 is normalized to the "default’ best-choice value in both GADGET–2, P-SPH, and GIZMO.

ArtCondConstant: Multiplies the dimensionless dissipation switch for the artificial conductivity term. Should be <1 to minimize diffusion (larger values make test problems look good, but actually substantially degrade your ‘effective resolution’).

ViscosityAMin: Minimum value of the artificial viscosity away from shocks. This can be zero, which will make some test problems look good, but will lead to serious errors (particle inter-penetration) in strong shocks when there is low resolution or particle disorder. Experimentation suggests at least a value =0.025–0.05 is required for stability in realistic problems. See Cullen & Dehnen 2010 for a description of this parameter (note they sometimes use zero in their test problems).

ViscosityAMax: Maximum artificial viscosity in shocks. This should be >1 to ensure shocks are well-captured; trial-and-error suggests 2 is a robust value.

ArtificialResistivityMax: As with the standard hydrodynamics, if we run the code in SPH mode, we require additional “artificial dissipation” switches to ensure numerical stability for MHD. These parameters (ignored if the code is not run in SPH mode) reflect the improved SPH-MHD “artificial resistivity” switches developed by Tricco+Price (see their code papers for details). This parameter functions analogously to ViscosityAMax, and sets the maximum (dimensionless) artificial resistivity values.

9. Snapshot & Initial Condition Files

Initial Conditions (Making & Reading Them)

File Formats and Necessary Fields

GIZMO initial conditions are exactly identical to GADGET. Any GADGET-compatible IC should be trivially useable in GIZMO, and vice versa.

The possible file formats for initial conditions are the same as those for snapshot files, and are selected with the ICFormat parameter. In fact, any valid snapshot file is also automatically a valid IC file. However, only the blocks up to and including the gas temperature (if gas particles are included) need to be present (so an IC file can have significantly less information than a full snapshot file). The IC file must include the initial positions, masses, and velocities of all particles, and [for gas] the initial internal energies. That information, plus the relevant header data (which you can see an example of in the “make_IC.py” file in the initial_conditions/ folder of the code) is all you need for an IC file. Things like gas densities, kernel lengths and all further blocks need not be provided. In preparing initial conditions for simulations with gas particles, the temperature block can be filled with zero values, in which case the initial gas temperature is set in the parameterfile with the InitGasTemp parameter. However, even when this is done, the temperature block must still be present. Note that the field Time in the header will be ignored when GIZMO is reading an initial conditions file. Instead, you have to set the time of the start of the simulation with the TimeBegin option in the parameterfile.

For details on actually reading the ICs (feeding them into the simulations), see the section in this Guide on Compiling & Using the Code (which walks you through how to actually read an IC into GIZMO). The section below on snapshots has much more detailed information on the actual file formats which can be used for GIZMO ICs.

Generating Initial Conditions Files

A large number of codes exist which can generate IC files for different types of GIZMO problems (again, anything that will generate ICs for GADGET will work for GIZMO). Here are (just a few) examples to get you started:

Note that in many cases, even if you can easily build an ‘approximate’ IC file which is pretty good, it may have some nasty non-equilibrium or initially transient features, or may have some nasty geometric features (like preferred grid directions, if you lay the particles on a lattice). It is often desirable to ‘relax’ the system into equilibrium before running your simulation. This can be especially important in methods like MFM or SPH, if there are sharp density gradients but fixed particle masses, since the density gradient depends on the spatial distribution/positions of the particles. It is useful in general whenever the ICs contain very sharp gradients or discontinuities – remember, in any numerical method, discontinuities have formally un-defined gradient and convergence properties (i.e. the gradient you get will be entirely determined by numerical error or slope limiter or similar terms). As a result, the behavior is ill-defined. What you should always do in such situations is relax the initial mesh so it does what you actually want it to do. Another example where this is important is hydrostatic equilibrium systems – for example, a simulation of a polytrope (like a star). At the ‘surface’, the sharp falloff of density means that if you just took a lattice of mesh-generating points and cut it, you would get a lot of nasty ‘edge effects’ (and moreover, if your equilibrium is very sensitive, then the difference between the back-of-the-envelope estimate of the density and the discrete, numerical calculation of the density or its gradient can actually introduce features in your simulation).

To deal with most or all of these cases, you can relax the mesh using the HYDRO_GENERATE_TARGET_MESH flag. This can be used with any hydro method. If enabled, you must provide the functions return_user_desired_target_density and return_user_desired_target_pressure, which are defined in the very top of the eos/eos.c file. The code will then attempt to evolve towards these ICs, using the actual in code definitions of density, gradients, force calculations, etc. This is accomplished by re-setting the pressure (and corresponding internal energies) in each timestep to a value \(P = P_{target}(\rho/\rho_{target})\) – in other words, if the density is locally above/below the target value, it will over/under-pressurize the region to move the particles. Note that if you set up your target pressure to have gradients, there must be a restoring force designed to balance them. For example, to set up hydrostatic equilibrium with a pressure gradient, you must use this flag, but also include e.g. a gravitational force, which can be solved via self-gravity or imposed analytically with the appropriate parameter choices (otherwise, if you just set up a pressure gradient with no restoring force, the system will blow up, as it should!). By simply setting the ‘target’ pressure and density to constant values, you’ll generate a glass-type mesh. A couple examples are written in these functions to get you started. Obviously, if your ICs also depend sensitively on other dynamical variables (if, for example, magnetic pressure is critical for your ICs), then you can trivially write similar functions for those variables as well and set them in this step. Also note this will obviously generate some motion of the mesh; if you wish that to be damped, to arrive at something non-moving, find the line with this compiler flag in kicks.c and un-comment it (that will enable velocity damping).

Building Self-Gravitating Body ICs (Planets, Asteroids, Stars, Compact Objects): the HSE IC builder

For self-gravitating fluid (or fluid-like) bodies whose interior structure is set by hydrostatic equilibrium against a tabulated equation of state – terrestrial and giant planets, planetesimals/asteroids/rocks and other solid bodies built on Tillotson or M-ANEOS tables; proto-stars, main-sequence stars, and white dwarfs evolved with the appropriate stellar EOS; neutron stars when combined with the nuclear-network EOS modules; brown dwarfs and substellar objects through the Chabrier-Debras H/He tables; or any other isolated body where a 1-D HSE profile defines the desired initial state – GIZMO ships a dedicated end-to-end Python IC-builder suite in initial_conditions/eos_tools/. The intent is to spare you the cottage-industry of hand-rolling a rejection-sampled sphere, a separate adiabatic-shooter, a separate placement script, and a separate u-correction pass for every new body. The suite carries each step end-to-end with a single uniform interface, and the recipe it implements has been validated against the SPH-planet literature to give percent-level residuals from analytic HSE for layered Earth-mass bodies.

The modules, all in initial_conditions/eos_tools/:

The canonical recipe – which is what we use in our own production planet ICs and recommend you start from – looks like:

from initial_conditions.eos_tools.eos_dispatch     import Material
from initial_conditions.eos_tools.hse_solver       import Zone, solve_hse
from initial_conditions.eos_tools.shell_placement  import glass_relax, atmosphere_shell, merge_placements
from initial_conditions.eos_tools.sph_density      import correct_u_for_sph_density
from initial_conditions.eos_tools.build_layered_body import write_hdf5

# Earth-like: forsterite mantle (68% by mass) + iron core (32%)
forsterite = Material.sesame("forsterite", "test/aneos_giant_impact/forsterite.sesame",
                              composition_type=0, Cv=1.0e7)
iron       = Material.sesame("iron",       "test/aneos_giant_impact/iron.sesame",
                              composition_type=1, Cv=4.5e6)
zones = [Zone(forsterite, inner_mass_frac=0.32, T_profile="adiabatic"),
         Zone(iron,       inner_mass_frac=0.00, T_profile="adiabatic")]

prof   = solve_hse(M_total=5.972e27, T_surface=2000.0, P_surface=1e6, zones=zones)
body   = glass_relax(prof, zones, n_particles=5000, n_sweeps=30)
atmo   = atmosphere_shell(prof, zones, n_particles=500, rho_atmo=1e-4, T_atmo=2000.0,
                           rho_taper_power=5.0, equal_mass_to=body)
placed = merge_placements(body, atmo)
correct_u_for_sph_density(placed, prof, zones, des_num_ngb=32)
write_hdf5("earth_ics.hdf5", placed, prof, box_size=10*prof["R"])

The same five-line core (solve_hse \(\to\) glass_relax \(\to\) atmosphere_shell \(\to\) correct_u_for_sph_density \(\to\) write_hdf5) handles any of the body classes named above; only the Material choices and the Zone list change. For a single-material body (a homogeneous basalt asteroid, a neutron star with a single nuclear EOS region, a brown-dwarf-like CD21 H/He envelope) the zones list is length one. For a multi-layer body (terrestrial planet with core+mantle+crust, a stratified giant planet, a stellar envelope on a compact-object core) add zones; the solver carries the material-boundary \(T\) continuity automatically. The atmosphere shell is optional but strongly recommended for any vacuum-boundary body in SPH/MFM; we leave it off for periodic-box compression tests.

The following test problems use the builder or its EOS tables, and can be cloned as starting templates for your own setup:

Multi-material ICs label each element’s material with the per-particle HDF5 “CompositionType” block. The code reads it from the IC and keeps it when it lies in the valid material-id range (values outside that range are reset to slot 0, the single sanitize point before any solid-EOS dispatch). For ICs that assign composition at run time instead, disable the read with IO_COMPOSITIONTYPE_NOT_IN_ICFILE (auto-enabled by GRAIN_FLUID_PROMOTION).

The builder is intentionally Python-only and run at IC construction time; no compile-time flags are needed for the builder itself, only for the GIZMO EOS_TILLOTSON / EOS_ANEOS / NUCLEAR_NETWORK / etc. solver code that consumes the resulting IC. A quantitative HSE check for a new body class – a run of several dynamical times with bounds on the surface radial velocity and the density-profile drift – is straightforward with the builder’s residual diagnostics and test/hse_earth_smoke/analyze_hse.py, and should be defined for the specific science application.

Snapshots

Un-Formatted Binary Format

The un-formatted binary snapshot format is also exactly the same as GADGET (up to the addition of certain output fields when additional physics is active in the simulations). If you are using this (usually for historical reasons), you should check the GADGET users guide, or you can just get the chapter specific to the snapshot format here.

Unless you have very good reason for using un-formatted binary (historical compatibility for some very special codes), you almost certainly DO NOT want to use this. Unformatted binary means that if even one byte is flipped, it can be difficult or impossible to recover data. Moreover, if your “snapshot reader” doesn’t know exactly the format (to the byte) and ordering of the data, it will crash or give errors (which means running the code with slightly different options, which will produce different outputs, leads inevitably to in-ability to use the same analysis codes on the simulations). For those reasons this is maintained only for legacy purposes but users are STRONGLY advised to use HDF5 wherever possible.

HDF5 Format

If the hierarchical data format is selected as format for snapshot files (always recommended!) or initial conditions files, GIZMO accesses files with low level HDF5 routines. Advantages of HDF5 lie in its portability (e.g. automatic endianness conversion), generality, ease of reading (different blocks in files appear as meta-data, so if your snapshot includes some extra data from some additional physics, which isn’t expected by the routine you’re using for snapshot-reading, it won’t crash the code or give errors!). But the fields expected/required in a GADGET HDF5 file are preserved in GIZMO, so again, everything is cross-compatible between the codes. There are also a very large number of tools available to easily manipulate or display HDF5 files (for example, the standard python library h5py). A wealth of information about HDF5 can be found on the HDF5 Project Website.

GIZMO stores the snapshot file data in several data groups, which are organized as follows:

/Header  
/Type0/Coordinates  
/Type0/Velocities  
/Type0/ParticleIDs  
...  
/Type1/Coordinates  
/Type1/Velocities  
...

The Header group only contains attributes that hold the fields of the file header in the standard file format of GADGET–2 (see the chapter linked above for details). As noted a couple times above, if you compile GIZMO using HDF5 versions 1.8 or later, you may need to use the flag -DH5_USE_16_API in the GIZMO compile in order to preserve backwards compatibility with the APIs used in code.

Units

Remember, the units of the code are specified by you, the user, in the parameterfile, when you run the code. The outputs will, of course, be in these units, and the inputs are expected to also be in these units. However sometimes it isn’t obvious (in cosmological simulations) whether the units are physical or not. We therefore provide some simple references below.

The “Standard” code units, expected for both initial conditions and snapshots, in a non-cosmological snapshot, are::

h = H0 / (100 km/s/Mpc) = HubbleParam = (params.txt default = 0.7)  
MASS_code = UnitMass_in_g / h = (params.txt default = 10^10 h^-1 M_sun)  
LENGTH_code = UnitLength_in_cm / h = (params.txt default = 1 h^-1 kpc)  
VELOCITY_code = UnitVelocity_in_cm_per_s = (params.txt default = 1 km/s)  
TIME_code = LENGTH_code/VELOCITY_code = (params.txt default = 0.978 h^-1 Gyr)  
INTERNAL ENERGY_code = VELOCITY_code^2 = (params.txt default = (km/s)^2)  
DENSITY_code = MASS_code/(LENGTH_code^3) = (params.txt default = 6.77e-22 h^-2 g/cm^3)  
MAGNETIC_FIELD_code = UnitMagneticField_in_gauss = (params.txt default = 1 Gauss)  
DIVERGENCE_DAMPING_FIELD_code = MAGNETIC_FIELD_code * VELOCITY_code

For each of the above “params.txt default” refers just to the default given in the example parameterfile that comes with the source code (“params.txt”) in the “scripts” folder. The actual value is set by the variable in the parameterfile as noted. The ambiguity here is the “little h” factor, which is common as a historical note in cosmological simulations. If your simulation is non-cosmological, easiest is to set “h=1” and nothing will care (you don’t care about the Hubble flow!) so you don’t need to keep track of it. Otherwise it does require some care. So for example, if I set UnitLength_in_cm equal to 3.085678e21 in the params.txt file (1 kpc), and HubbleParam equal to 0.71, then if something in my snapshot with length units (say, the “x” coordinate position of a particle) has value 1.3, the physical value is given by: snapshot_value x LENGTH_code = 1.3*3.085678e21 cm / 0.71 = 1.83 kpc.

In cosmological simulations, many of the units are co-moving, and this needs to be accounted for::

a_scale = 1/(1+z) = scale factor, also the "time unit" of the code  
LENGTH_physical = LENGTH_code * a_scale  
MASS_physical = MASS_code  
VELOCITY_physical = VELOCITY_code * sqrt(a_scale)  
DENSITY_physical = DENSITY_CODE / a_scale^3  
KERNEL_LENGTH_physical = KERNEL_LENGTH_CODE * a_scale  
MAGNETIC_FIELD_physical = MAGNETIC_FIELD_code  
INTERNAL_ENERGY_physical = INTERNAL_ENERGY_code  
MAGNETIC_FIELD_physical = MAGNETIC_FIELD_code (note that *in* the code, co-moving units are used B_code=a_scale^2 B_phys, but in outputs these are converted to physical)  
DIVERGENCE_DAMPING_FIELD_physical = DIVERGENCE_DAMPING_FIELD_code (again, in-code, psi_physical = a_scale^3 psi_code)

So here, the statement LENGTH_physical = LENGTH_code * a_scale means that the code output in the snapshot file must be multiplied by both the specified units LENGTH_code (just as described above for non-cosmological snapshots) and the correction factor a_scale to go from co-moving to physical units. So let’s take our example from above with length “1.3”. We do the same thing as before, get 1.83 kpc – but this is co-moving. Let’s say this snapshot is at scale-factor a_scale = 0.2 (redshift 4); the physical length is therefore 1.83 kpc * a_scale = 0.366 kpc.

Also note that as stated above the “time unit” of the code being “a_scale” means that things like the beginning/end time of the simulations, the timestep size, the formation times of star particles and current “time” saved in snapshots, will be scale factor, in cosmological runs. If you’re not sure whether a time-type quantity is actually time or scale-factor, usually a quick sanity check is all that is needed (since scale factor only runs between 0 and 1). But remember, this is only for cosmological runs, for non-cosmological runs, everything is always in physical units.

One more minor note: the “kernel length” (or, for SPH, the “smoothing length”) saved in the snapshot is the maximum radius out to which neighbors are searched (the ‘radius of compact support’), often denoted something like \(H_{\rm sml}\) in SPH codes or \(R_{\rm ngb}^{\rm max}\) in finite-element methods. The resolution scale, what is called \(h\) (the kernel length) or \(\epsilon\) (the softening length) in the standard convention in the literature, is the scale length/standard deviation of the equivalent Gaussian or Plummer kernel. For a standard cubic-spline kernel kernel with effective interaction-kernel neighbor number 32, this is =KernelLength/2 or =KernelLength/2.8 (equivalent Plummer sphere); if you use the quintic kernel (enabled at compile-time), this is =KernelLength/3 (equivalent Gaussian). Something more accurate still like Dehnen & Aly’s definition of the minimum stable soundwave wavelength gives ~KernelLength/3 for our standard kernel. A simpler definition which is more robust and easier to compare with other codes is to take the equivalent inter-neighbor separation, i.e. the equivalent cell side-length in a Cartesian regular rectilinear grid code. This gives \(\Delta x_{i} = (m_{i}/\rho_{i})^{1/N_{\rm D}}\) where \(m_{i}\) is the mass of the resolution element, \(\rho_{i}\) the local density (saved in snapshots), and \(N_{\rm D}\) is the number of spatial dimensions being explicitly evolved in the code.

Reading Snapshots (Examples)

First note that there are many, many public codes and libraries (for python, IDL, C, and other languages) which can read and interact with GIZMO snapshots (or GADGET snapshots, which follow the same format). This includes for example YT, PynBody, PyGadgetReader, SPHGR, PyGad, and more general tools like MESHOID, vaex, Glue, TOPCAT. Even more examples are given in the “Additional Resources” section below. But it is important to understand directly what is in your snapshots, in case these codes are not appropriate for your analysis (or you are running simulations with different physics).

For most snapshot analysis we recommend MESHOID, a dedicated mesh-free analysis package. GIZMO also ships lightweight built-in snapshot readers in the top-level analysis/ directory: load_from_snapshot.py and readsnap.py (plus the legacy IDL readsnap.pro in analysis/legacy_idl/). To use them, put analysis/ on your Python path and import directly, e.g. from load_from_snapshot import load_from_snapshot. load_from_snapshot is the most up-to-date reader and includes fixes for unit conversions and modern field names.

The older routines readsnap.py and readsnap.pro (Python and IDL, respectively) are not very fast or efficient, but demonstrate how to read a variety of file types in full generality, including the legacy pure-binary format. The more modern load_from_snapshot.py is far more efficient and can read any field in the snapshots by name, but only works for HDF5 outputs. Note that these routines are specific to snapshots from simulations run including certain specific physics, and may have to be modified for the specific physics and conditions under which you run your simulation.

The routines load_from_snapshot.py and readsnap.py can be called by standard python installations, below we discuss how to use them, for a specific example, where we assume one is trying to read the outputs of a cosmological simulation with galaxy formation, star formation, cooling, and super-massive black hole physics enabled at compilation time. You should use this as a guide, not as an exact template for your uses of the code!

For specific instructions on using either routine, you can open the raw python file and read the help text, or from within python, import the files and then type help(load_from_snapshot) or help(readsnap) and it should display that same help text – this gives a full description of the syntax and variable names you need to pass to the program. For example, you can copy the entire data block for elements of a given type with the syntax P=readsnap(snapshot_directory,snapshot_number,element_type,optional_flags...) or load a specific set of values, say the element coordinates, with the syntax xyz_coordinates=load_from_snapshot('Coordinates',element_type,snapshot_directory,snapshot_number,optional_flags...). In both cases you feed it the parent directory of the snapshot files, the snapshot number (as an integer), the desired particle/element type (more below), and a number of optional flags if desired (e.g. you can tell it to convert the units from co-moving to physical, or not, as you choose). For load_from_snapshot.py, you load a specific field from the snapshots by passing the name of that field (more below, with examples of specific fields) – it can read any field that exists in the snapshot files, but can only do this by relying on the HDF5 structure. For readsnap.py it pre-loads a large number of common fields into a global structure P which you can then access with a shorthand name as P['shorthand_name'] (this takes more time and memory and cannot find fields it is not pre-programmed to look for, but it can read the un-formatted binary files and grab everything at once).

Each file contains many fields. If the file is in the old binary format, you need to know exactly ahead of time what the fields are and what order they are in (the reader routine will do this). Much easier is if the files are in HDF5 format. This is always recommended for snapshot outputs.

Note that you should always be careful and double-check units. Most outputs will be in code units, but some will (for historical or physical reasons) use different units. Routines like load_from_snapshot will attempt to help you by giving you the option to automatically convert co-moving quantities into physical quantities: but make sure you know what the script is doing (and its not incorrectly converting something it should not). Likewise make sure you can get the same answer by reading directly from the HDF5 file, so you know your script is doing the right thing.

All files include a header block with (at minimum) the following quantities/flags:

Time = time at the moment of the snapshot: for non-cosmological runs, this is an actual time, in h^-1 Gyr; in cosmological runs it is the SCALE FACTOR (=1/(1+z))

NumFilesPerSnapshot = number of parts/subfiles the snapshot is broken into  
MassTable = 6-element table with the fixed masses for each of particle types [0,1,2,3,4,5]: if a value is =0, the values are given as an array for each particle (they may not all be exactly equal) -- so usually you won't get masses here  
Flag_Sfr = flag that says whether star formation is on (=1) or off (=0) for the run  
Flag_Cooling = flat that says whether cooling is on or off  
Flag_Feedback = ignore me  
Flag_StellarAge = flag that says whether star particles have ages in snapshot  
Flag_Metals = flag that says whether metals are included in the run 

NumPart_Total = 6-element array with the *total* number of particles of each type [0,1,2,3,4,5] in the snapshot  
NumPart_ThisFile = array (as numpart_total) with the number of particles *in the file* (for multi-file snapshots)

BoxSize = side-length of periodic box for cosmological runs  
Omega0 = omega matter for the run (if cosmological)  
OmegaLambda = omega lambda for the run (if cosmological)  
HubbleParam = hubble parameter "h" for the run  
Redshift = redshift of snapshot

Newer versions of the code include a wide range of additional flags in the header, provided the appropriate physics is enabled at compile time (and that you are using HDF5). This makes it easier to figure out exactly how your run was set up, if you only have the snapshots. This includes, for example:

GIZMO_version = version (year) of GIZMO code

ComovingIntegrationOn = was this simulation run in co-moving/cosmological form (1=yes)  
Omega_Matter = cosmological omega matter (if ComovingIntegrationOn)  
Omega_Lambda = cosmological omega lambda (if ComovingIntegrationOn)  
Omega_Baryon = cosmological omega baryon (if ComovingIntegrationOn)  
Omega_Radiation = cosmological omega radiation (if ComovingIntegrationOn)

UnitMass_In_CGS = user-specified unit mass in cgs (from parameterfile)  
UnitVelocity_In_CGS = user-specified unit velocity in cgs (from parameterfile)  
UnitLength_In_CGS = user-specified unit length in cgs (from parameterfile)  
Internal_UnitB_In_Gauss = internal code B-units (not what is in the snapshot, but useful to check if desired setup is actually done, if MAGNETIC on)  
Gravitational_Constant_In_Code_Inits = gravity constant G in code units

Minimum_Mass_For_Cell_Merge = minimum mass below which cells are merged  
Maximum_Mass_For_Cell_Split = maximum mass above which cells are split

Fixed_ForceSoftening_Keplerian_Kernel_Extent = 6-element table with the in-code kernel-radius-of-compact-supprt of the gravitational force softening value for each particle type, at the time of the snapshot  
Kernel_Function_ID = value of KERNEL_FUNCTION, specifying the kernel function  
Effective_Kernel_NeighborNumber = target effective number of cells inside the radius of compact support of one kernel

Subfind_FOFLink_NeighborNumber = neighbor number for FOF linking for subfind (if SUBFIND is enabled)

Solar_Abundances_Adopted = value of solar abundances assumed in code for the list of metals evolved (METALS on)  
Metals_Atomic_Number_Or_Key = atomic number of each metals species evolved explicitly in-code; values <1 correspond to special keys, i.e. 0=all/total metallicity, -1=R-process tracer fields, -2=age-tracer fields, -3=starforge feedback tracer fields

Radiation_RHD_Min_Bin_Freq_in_eV = if radiation-hydrodynamics is enabled, with outputs saved, this saves the minimim frequency of the explicitly-evolved radiation 'bins' which are actually saved in the code/snapshot outputs (should match the number of elements saved there, so you have a 1-to-1 correspondence), in units of eV (so this is h*nu). note a couple special keys: -1 is for RT_GENERIC_USER_FREQ, since this has no defined frequency range, and -2 for RT_FREEFREE, which evolves the bolometric (all-wavelength-integrated) free-free  
Radiation_RHD_Max_Bin_Freq_in_eV = as Radiation_RHD_Min_Bin_Freq_in_eV, but the upper limits of the bins/intervals (same special keys)

... and many more, depending on the physics enabled ...

In the current versions of the code, almost every parameter of interest specified in the parameterfile will be saved to the header (e.g. if you enable cosmic rays and specify the diffusion coefficient in the parameterfile, that is saved here as CosmicRayDiffusionCoeff_at_GV_CodeUnits). Other useful information includes e.g. for spectrally-resolved cosmic ray methods the properties (species, charge, rigidity, etc) of each bin, or the age range of each bin for the stellar age tracer modules, etc. Please check the header files for your simulation outputs, they contain a tremendous amount of useful information!

Then, for each particle type, there are a set of structures/arrays; these are sub-divided by PARTICLE TYPE. particle type=0 is always gas resolution elements (we really shouldn’t call them ‘particles’ since the code treats them as finite volume elements or even explicit, traditional fixed-grid ‘cells’ in some modes, but for the sake of simplicity, since the N-body particles are indeed ‘particles’, I’ll use that term throughout this guide). The other particle types are not gas. What we chose them to represent is arbitrary and depends on the compile-time flags in the code. For the default example we are discussing here:

0 = gas/fluid (the cells that feel local fluid-dynamic forces)  
1 = high-resolution dark matter (in e.g. galaxy/cosmological simulations)  
2,3 = 'dummy' collisionless particles (e.g. low-res dark-matter particles in cosmological runs; pre-existing "disk" and "bulge" star particles in the non-cosmological runs, dust grains or cosmic ray particles in simulations with explicit grain dynamics or MHD-PIC simulations)  
4 = stars formed during the simulation (in e.g. simulations with non-sink 'star particle' formation like galaxy sims, otherwise another collisionless species)  
5 = sink particles (e.g. black holes in galaxy simulations, or protostars or star particles in simulations with sink-particle star formation, or if no label is given, this is another dummy collisionless particle)

For each particle/cell, then, there are a wide range of data the routine will attempt to read. See the routine itself to see how these are embedded in the snapshots and extracted. Here we will outline what each quantity is. Each entry in bold below refers to the name of the HDF5 block, containing the data. The term in parentheses (P[‘x’]) denotes the name of the block (usually some shorthand) which is returned by the routine readsnap.py (this can be trivially modified by the user). If you use load_from_snapshot.py, each data structure can be retrieved by asking for it with the proper name used by the HDF5 file. Here we give the description of what youre looking at.

Coordinates (output by readsnap.py routine as P[‘p’]): a [N,3] array where N is the number of particles (of the given type). Elements [i,0], [i,1], [i,3] give the x, y, z (respectively) coordinate positions of particle “i” in code units

Velocities (P[‘v’]): [N,3] array (same format as Coordinates) with x, y, z particle velocities (in code units)

ParticleIDs (P[‘id’]): [N] array with the unique integer ID for every particle. these are the same across snapshots, and can be used to link particles across time. Note that particles are NOT listed in the same order in each snapshot (the ordering in these arrays has to do with the positions of the particles in the simulation at that time). So you need this to link them. Note also that star particles (type=4) will retain the id of their “parent” gas particle (type=0), so you know the particle from which they formed.

Note that if particle splitting/merging is enabled, as it is by default in the code, then two additional ID variables will be present, to distinguish split particles from one another (the old method using bit-flipping is not able to handle multiple generations). ParticleChildIDsNumber is an [N] array with a unique sub-number for each parent ParticleID – this distinguishes the different split particles so you can follow them subsequently in time. The combination of ParticleID and ParticleChildIDsNumber is always unique. ParticleIDGenerationNumber is an [N] array saying which generation the particle comes from (how many generations of splits have occurred from the ICs to produce the given particle, along that particles ancestor history).

Masses (P[‘m’]): [N] array of particle masses (in code units)

Softening_KernelRadius: [N]-array (only included if OUTPUT_SOFTENING is set; written for every particle type): the gravitational force-softening length of each element, given as the radius of compact support of the softening kernel (from ForceSoftening_KernelRadius), in code length units. With adaptive softening (ADAPTIVE_GRAVSOFT_FORGAS/ADAPTIVE_GRAVSOFT_FORALL) this varies from element to element.

AGS-KernelRadius: [N]-array (included whenever adaptive kernel lengths are computed for non-gas particles: ADAPTIVE_GRAVSOFT_FORALL, DM_SIDM, DM_FUZZY, CBE_INTEGRATOR, or grain-collision modules): the radius of compact support of each such particle’s kernel, set by AGS_DesNumNgb, in code length units.

AGS-Density: [N]-array (only included if DM_FUZZY or CBE_INTEGRATOR is set): the kernel-estimated density of the adaptively-softened particles, in code units. Under CBE_INTEGRATOR it is the particle mass divided by its kernel volume.

AGS-QuantumPotentialQ: [N]-array (only included if DM_FUZZY is set): the quantum potential of the scalar-field dark matter at each particle, evaluated from the kernel density and its first and second derivatives, in code units of specific energy.

WavefunctionPsi-Real, WavefunctionPsi-Imag: [N]-arrays (only included if DM_FUZZY is set to 1 or 2, the direct Schroedinger-Poisson integrators): the real and imaginary parts of the scalar-field wavefunction carried by each particle.

AGS-Zeta: [N]-array (only included if AGS_OUTPUTZETA is set): the adaptive-softening correction term \(\zeta\) of each adaptively-softened particle.

RefinementFlag: [N]-array of integers (64-bit; only included if SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM_TAG_ANCHOR is set; written for every particle type): the tag read from the ICs. Elements with value 1 define the anchor point (the mass-weighted center or the densest tagged element) that the nuclear-zoom refinement follows. The code carries the value through unchanged and writes it back so that restarts from a snapshot keep it.

VlasovMoments: [N, N_basis*N_moments]-array (only included if CBE_INTEGRATOR is set; the block is written for every particle type, but it is only evolved for CBE_INTEGRATOR_PARTICLETYPES): the moments of each basis function in the collisionless-Boltzmann (CBE) representation of the distribution function, in code units. \(N_{\rm basis}\) is the value of CBE_INTEGRATOR. Element [i, N_moments*k + m] is moment m of basis function k. Moment 0 is the mass in that basis function. The next \(N_{\rm dim}\) moments are its momentum relative to the element velocity. With CBE_INTEGRATOR_SECONDMOMENT, the remaining moments are the independent components of the symmetric second-moment tensor (\(m\,(v_{a}v_{b} + \sigma^{2}\delta_{ab})\) in the relative frame). \(N_{\rm moments} = N_{\rm dim}+1\) by default, or 3/6/10 in 1D/2D/3D with CBE_INTEGRATOR_SECONDMOMENT. The same block is read back from ICs when it is present.

Certain quantities are specific to only gas particles. These include:

InternalEnergy (P[‘u’]): [N]-element array, particle specific internal energy (internal energy per unit mass, in code units). units are physical

Note: to convert internal energy to TEMPERATURE, use something like the following: 

Temperature = mean_molecular_weight * (gamma-1) * InternalEnergy / k_Boltzmann  
k_Boltzmann is the Boltzmann constant  
gamma = 5/3 (adiabatic index)  
mean_molecular_weight = mu*proton_mass (definition)  
mu = (1 + 4*y_helium) / (1+y_helium+ElectronAbundance)  (ignoring small corrections from the metals)  
ElectronAbundance defined below (saved in snapshot)  
y_helium = helium_mass_fraction / (4*(1-helium_mass_fraction))  
helium_mass_fraction is the mass fraction of helium (metallicity element n=1 described below)

Density (P[‘rho’]): [N]-element array, code-calculated gas density, at the coordinate position of the resolution element. To obtain the effective volume of an element, to convert an energy into an energy density, for example, you simply take mass divided by density, i.e. by definition \(V_{i} = m_{i}/\rho_{i}\).

KernelMaxRadius (P[‘h’]): [N]-element array, kernel radius of compact support for interaction face-element definition (called the “smoothing length” for the sake of historical compatibility with SPH in older versions of GIZMO, even though there is no smoothing within the kernel for the MFM/MFV methods).

ElectronAbundance (P[‘ne’]): [N]-element array (only included if Flag_Cooling>0): this is the mean free-electron number per proton (hydrogen nucleon); averaged over the mass of the gas particle

NeutralHydrogenAbundance (P[‘nh’]): [N]-element array (only included if Flag_Cooling>0): neutral hydrogen fraction (between 0 and 1). With Grackle chemistry (COOL_GRACKLE_CHEMISTRY>0) this is instead the HI mass fraction of all gas (smaller than the neutral fraction per hydrogen by the hydrogen mass fraction).

If OUTPUT_COOLRATE_DETAIL is set together with COOLING, the individual terms of the cooling function are saved for each gas element (Type=0). They are the values left by the last evaluation of the cooling function for that element. All of them are in cgs units of erg cm${3}$ s${–1}$: each is a volumetric rate divided by \(n_{\rm H}^{2}\), the same normalization used internally for \(\Lambda\). Multiply by \(n_{\rm H}^{2}\) (in cm${–3}$) to get erg cm${–3}$ s${–1}$. (The total, CoolingRate, is written under the same flag.)

HeatingRate: [N]-array: total heating rate \(\Gamma\) (sum of all heating terms), positive for heating.

NetHeatingRateQ: [N]-array: net rate \(Q = \Gamma - \Lambda\) (positive = net heating). With COOL_LOW_TEMPERATURES (and not COOL_LOWTEMP_THIN_ONLY), it is limited by the optically-thick blackbody cap. It does not include the hydrodynamic work term.

HydroHeatingRate: [N]-array: the hydrodynamic work term (the hydro-solver \(du/dt\), converted to the same per-\(n_{\rm H}^{2}\) units), which is added to \(Q\) in the implicit cooling solve. It is set only when COOLING_OPERATOR_SPLIT is not set.

MetalCoolingRate: [N]-array: metal-line cooling from the species-by-species tables (only evaluated if COOL_METAL_LINES_BY_SPECIES is set and a UV background is present).

DustCoolingRate: [N]-array (only included if GALSF_ISMDUSTCHEM_MODEL and GALSF_ISMDUSTCHEM_HIGHTEMPDUSTCOOLING are also set): cooling of hot gas through collisions with dust grains.

PElecHeatingRate: [N]-array: photoelectric heating rate, positive for heating. It is only evaluated when GALSF_FB_FIRE_RT_LONGRANGE, RT_PHOTOELECTRIC or RT_ISRF_BACKGROUND is set, and only for \(T<10^{6}\) K with a non-zero FUV field.

MolecularMassFraction: [N]-array (only included if OUTPUT_MOLECULAR_FRACTION is set; gas only): the code estimate of the molecular (H${2}$) mass fraction of the hydrogen, between 0 and 1. It is set by the cooling function, so it needs COOLING. If COOL_MOLECFRAC_NONEQM is set, the value written is instead the explicitly-evolved H${2}$ fraction of the neutral hydrogen (multiply by the neutral fraction to get the fraction of all hydrogen). Detailed chemistry modules (Grackle, CHIMES) output their own species, below.

GrackleHI, GrackleHII, GrackleHM, GrackleHeI, GrackleHeII, GrackleHeIII (if COOL_GRACKLE_CHEMISTRY>=1); GrackleH2I, GrackleH2II (if COOL_GRACKLE_CHEMISTRY>=2); GrackleDI, GrackleDII, GrackleHDI (if COOL_GRACKLE_CHEMISTRY>=3): each is an [N]-array (gas only) with the abundance of that Grackle species (HI, HII, H${-}$, HeI, HeII, HeIII, H${2}$, H${2}{+}$, DI, DII, HD). Each value is the species mass density divided by the total gas density: a mass fraction of all the gas, not a fraction per hydrogen atom.

When CHIMES is active, ElectronAbundance and NeutralHydrogenAbundance are not written. The CHIMES species abundances replace them:

ChimesAbundances: [N, N_species]-array (gas only): the full CHIMES network abundances, \(n_{i}/n_{\rm H}\) for each species i (157 species for the full network), in the CHIMES species order (see the CHIMES documentation). If CHIMES_REDUCED_OUTPUT is set, this block is only written in snapshots whose number is a multiple of the parameter N_chimes_full_output_freq.

ChimesReducedAbundances: [N,4]-array (only if CHIMES_REDUCED_OUTPUT is set, and only in the snapshots that do not get the full ChimesAbundances array): the abundances \(n_{i}/n_{\rm H}\) of electrons, HI, H$_{2}$ and CO, in that order.

ChimesMu: [N]-array (gas only): the mean molecular weight computed from the CHIMES abundances (dimensionless).

ChimesColumnDensity: [N]-array (only included if CHIMES_NH_OUTPUT is set; gas only): the hydrogen column density in cm${–2}$ used for shielding in the chemistry solver. It is the Sobolev-type gradient estimate scaled by the CHIMES Shielding_length_factor and by the hydrogen mass fraction \(1-Z-Y\), so the non-equilibrium solve can be reproduced in post-processing.

ChimesFluxG0: [N, 8]-array (only included if CHIMES_STELLAR_FLUXES is set; gas only): the local stellar FUV (6–13.6 eV) flux in Habing units, split into the 8 stellar-age bins used by CHIMES. With CHIMES_HII_REGIONS, elements currently flagged as part of an HII region also include the HII-region contribution.

ChimesFluxIon: [N, 8]-array (only included if CHIMES_STELLAR_FLUXES is set; gas only): the local stellar ionizing photon flux in photons cm${–2}$ s${–1}$, in the same 8 age bins, with the same HII-region treatment as ChimesFluxG0.

SigmaEff: [N]-array (only included if CHIMES_NH_OUTPUT and OUTPUT_DENS_AROUND_NONGAS are both set; written for star particles, Type=4, plus Types 2 and 3 in non-cosmological runs): the effective gas surface density around the star in g cm${–2}$. It uses the same gradient-based column estimator applied to the gas density at the star’s position.

StarFormationRate (P[‘sfr’]): [N]-element array (only included if Flag_Sfr>0): star formation rate at that instant evaluated for the gas particle. units are solar masses per year, and this does include the effects of additional star formation thresholds in the code (virial, molecular, etc.).

DelayTime: [N]-array (only included if GALSF_SUBGRID_WINDS is set; gas only): the remaining time for which a gas element kicked into a sub-grid wind stays “decoupled”. While it is positive, the element cannot form stars and is skipped as a neighbor in the hydro force and gradient loops. It is set to WindFreeTravelMaxTimeFactor/\(H(a)\) at launch, counts down by the physical timestep, and is reset to zero once the element’s density drops below WindFreeTravelDensFac times the star formation threshold. Physical code time units.

DelayTime_HIIRegion_Cooling: [N]-array (only included if OUTPUT_DELAY_TIME_HII and GALSF_FB_FIRE_RT_HIIHEATING are both set; gas only): the remaining time for which the element is flagged as part of a compact HII region by the FIRE HII-region model, in physical code time units. It is set to at most 10 Myr when the element is ionized, and counted down in the cooling step. In newer versions (GALSF_FB_FIRE_STELLAREVOLUTION>2, or without that flag) a negative value means “expired this step; reset on the next one”.

Metallicity (P[‘z’]): [N,1+N_species]-element array (only included if Flag_Metals>0): metallicity of the gas. Metallicity units are mass fraction – so “solar” would be ~0.02 in total metallicity. Array element [i,n] gives the metallicity of the n-th species, for the i-th particle.

For our “standard” runs with METAL_SPECIES_COOLING enabled, N_species = 10, and the species are as follows:

n=0: "total" metal mass (everything not H, He)  
n=1: He  
n=2: C  
n=3: N  
n=4: O  
n=5: Ne  
n=6: Mg  
n=7: Si  
n=8: S  
n=9: Ca  
n=10: Fe

DustToGasRatio_Local: [N]-array (only included if OUTPUT_DUST_TO_GAS_RATIO is set. That flag is turned on automatically by RT_OPACITY_FROM_EXPLICIT_GRAINS, GALSF_ISMDUSTCHEM_MODEL or RT_INFRARED. The block is written for every particle type): the total metallicity \(Z\) (Metallicity[:,0]) multiplied by the local dust-to-metals ratio relative to solar, as returned by return_dust_to_metals_ratio_vs_solar. That ratio comes from the explicit grain opacity, the live dust model, the IR dust temperature, or the low-temperature dust model, whichever module is active. For non-gas particles the ratio is 1, so the value is just \(Z\). Multiply by the solar dust-to-metals ratio to get an absolute dust-to-gas mass ratio.

If the live ISM dust model GALSF_ISMDUSTCHEM_MODEL is active, the following are written for gas:

DustMetallicity: [N, N_elem+4]-array: the first N_elem = 1+N_species columns are the dust mass fraction of each element, in the same order and units as Metallicity (column 0 = total dust mass fraction, then He, C, N, O, Ne, Mg, Si, S, Ca, Fe). The last 4 columns are the fraction of the dust mass contributed by each source: gas-dust accretion, SNe Ia, SNe II, AGB winds. They sum to 1, or are all 0 when there is no dust.

DustSpeciesAbundance: [N, N_dustspecies]-array: the mass fraction of each tracked dust species (same mass-fraction units as Metallicity). The species follow the bits of GALSF_ISMDUSTCHEM_MODEL: silicates and carbonaceous for bit 1, then metallic iron (bit 2), the oxygen reservoir (bit 4) and iron inclusions (bit 8), each only if enabled.

DustMolecularSpeciesFractions: [N,2]-array (only if GALSF_ISMDUSTCHEM_GRAINSIZEEVO is not set): column 0 is the mass fraction of the gas in the dense molecular phase, and column 1 is the fraction of carbon locked in CO. Both are the sub-resolved values used by the dust routines.

DustBinNumbers: [N, N_dustspecies*N_bins]-array (only if GALSF_ISMDUSTCHEM_GRAINSIZEEVO is set; N_bins is its value): the number of grains in each size bin of each dust species in the element, divided by the header attribute UnitGrainNumber. Element [i, N_bins*s + b] is species s, bin b.

DustBinMasses: [N, N_dustspecies*N_bins]-array (same flag and ordering as DustBinNumbers): the total grain mass in each size bin, in code mass units.

MachNumber: [N]-array (only included if OUTPUT_MACH_NUMBER is set, which GALSF_ISMDUSTCHEM_GRAINSIZEEVO turns on automatically): the local resolution-scale turbulent Mach number used for sub-grid clumping in the grain-size evolution: \(|\nabla{\bf v}|\,\Delta x\) divided by the molecular sound speed. It is only computed when GALSF_ISMDUSTCHEM_GRAINSIZEEVO is active. (This is distinct from ShockMachNumber.)

If the run uses an SPH hydro solver, it also includes:

ArtificialViscosity: [N]-array (only included if SPHAV_CD10_VISCOSITY_SWITCH is set, which happens automatically for SPH unless SPH_DISABLE_CD10_ARTVISC is set or EOS_TILLOTSON, EOS_ELASTIC or EOS_ANEOS is used): dimensionless Cullen & Dehnen (2010) artificial viscosity parameter “alpha” of the gas particle, multiplied by its limiter (i.e. alpha times alpha_limiter). It does not include the Balsara switch, which is applied within the hydro sub-routine.

VelocityDivergence: [N]-array (only included if OUTPUT_VORTICITY or TURB_DRIVING is set; gas only): the kernel-estimated velocity divergence \(\nabla\cdot{\bf v}\) used for the kernel-size update, in code units (comoving: multiply by \(a^{-2}\) for physical, as the code does internally). It is not the higher-order gradient-matrix estimate, which is \(\mathrm{trace}\) of VelocityGradient.

DensityGradient: [N,3]-array (only included if OUTPUT_GRADIENT_RHO is set; gas only): the density gradient vector \(\nabla\rho\), in physical code units (mass/length${4}$).

VelocityGradient: [N,9]-array (only included if OUTPUT_GRADIENT_VEL is set; gas only): the full velocity gradient tensor, in physical code units (1/time; the Hubble flow is not added). Element [i, 3*a + b] \(= \partial v_{a}/\partial x_{b}\).

TurbulenceDriving: [N]-array (only included if TURB_DRIVING is set; gas only): the specific kinetic energy injected by the turbulent driving force, averaged over the interval since the last update of the driving phases. Code units of energy per unit mass per unit time.

TurbulenceDissipation: [N]-array (only included if TURB_DRIVING is set; gas only): the matching dissipation diagnostic, averaged over the same interval. It is only accumulated when the internal energy is reset by EOS_ENFORCE_ADIABAT or EOS_MHD_CORE_BAROTROPIC; otherwise it is zero.

If TURB_DIFF_DYNAMIC is active (localized dynamic Smagorinsky model):

DynSmagCoeff: [N]-array: the dimensionless dynamic Smagorinsky coefficient (\(C_{s}^{2}\)), limited to \([0,\)TurbDynamicDiffMax\(]\).

TurbDiffCoeff: [N]-array: the resulting turbulent diffusivity, TurbDiffusion_Coefficient\(\times C_{s}^{2}\,h_{\rm turb}^{2}\,|S|\), in physical code units (length${2}$/time).

DynamicError, DynamicErrorDefault: [N]-arrays (only included if OUTPUT_TURB_DIFF_DYNAMIC_ERROR is also set): the dimensionless relative error of the model fit to the Leonard stress tensor. It is the norm of the trace-free off-diagonal residual divided by the norm of the Leonard tensor. DynamicError uses the dynamically computed coefficient. DynamicErrorDefault uses a fixed coefficient of 0.05 for comparison.

If the run includes magnetic fields:

MagneticField: [N,3]-element array with B-field x,y,z components (units are Gauss and physical)

DivergenceOfMagneticField: [N]-array, Div[B] (i.e. \(\nabla\cdot{\bf B}\)) at particle position (useful for testing div-cleaning). Units are Gauss/code length, and are physical. A common dimensionless measure of the divergence (which ideally would be zero) is \(h\,|\nabla\cdot{\bf B}|/|{\bf B}|\), where \(h=V_{i}/2\), \((V_{i}/\pi)^{1/2}\), \((3V_{i}/4\pi)^{1/3}\) in 1D, 2D, and 3D, respectively, and \(V_{i}=m_{i}/\rho_{i}\) where \(m_{i}\) and \(\rho_{i}\) are the mass and density in physical units.

DivBcleaningFunctionPhi: [N]-array (only included if DIVBCLEANING_DEDNER and OUTPUT_BFIELD_DIVCLEAN_INFO are set): the scalar field \(\psi\) (phi) of the Dedner divergence-cleaning scheme. It is written with a factor \(a^{-3}\) and the same magnetic-unit conversion as MagneticField.

DivBcleaningFunctionGradPhi: [N,3]-array (same flags): the gradient of that field, written with a factor \(a^{-4}\) and the same magnetic-unit conversion.

MagneticFieldGradient: [N,9]-array (only included if OUTPUT_GRADIENT_MAG is set): the magnetic field gradient tensor in physical units of Gauss per code length. Element [i, 3*a + b] \(= \partial B_{a}/\partial x_{b}\).

ArtMagneticDissipation: [N]-array (only included if SPH_TP12_ARTIFICIAL_RESISTIVITY is set): the dimensionless artificial-resistivity coefficient \(\alpha_{B}\) of the SPH method: \(h\,|\nabla{\bf B}|/|{\bf B}|\), limited to between 0.005 and 0.1$\times$ArtMagDispConst.

OhmicResistivity, HallResistivity, AmbipolarResistivity: [N]-arrays (only included if MHD_NON_IDEAL and OUTPUT_MHD_RESISTIVITY are set): the Ohmic, Hall and ambipolar diffusion coefficients \(\eta_{O}\), \(\eta_{H}\), \(\eta_{A}\) used in the non-ideal induction equation, in physical code units of length${2}$/time. They are zero on the first timestep, and their sum is capped at \(10^{24}\) cm${2}$ s${–1}$.

If the run includes explicit radiation-hydrodynamics:

PhotonEnergy: [N,Nfrequencies]-array, containing for each of [N] elements the total radiation energy (in code units), integrated over the volume domain of that resolution element (i.e. the sum over particles gives the total radiation energy, not the specific radiation energy or energy density). The other dimension Nfrequencies corresponds to the number of frequency bins being evolved: each bin will output its own energy for every element.

PhotonFluxDensity: [N, 3*Nfrequencies]-array (only included if OUTPUT_RT_RAD_FLUX is set with a method that evolves the flux, RT_EVOLVE_FLUX, e.g. M1): the radiation flux vector of each frequency bin, per unit volume (the evolved, volume-integrated flux multiplied by \(\rho/m\), with \(\rho\) physical). Code units. Element [i, Nfrequencies*a + f] is component a (x,y,z) of bin f.

PhotonOpacity: [N,Nfrequencies]-array (only included if OUTPUT_RT_RAD_OPACITY and RADTRANSFER are set): the opacity \(\kappa\) of each frequency bin, in physical code units of length${2}$/mass.

EddingtonTensor: [N, 6*Nfrequencies]-array (written by default whenever RADTRANSFER is active, unless it is suppressed by the SINGLE_STAR_RT_DEFAULTS setup): the dimensionless Eddington tensor of each frequency bin. Element [i, Nfrequencies*c + f] is component c of bin f, with c = 0..5 in the order xx, yy, zz, xy, yz, xz.

IRBand_Radiation_Temperature: [N]-array (only included if RADTRANSFER and RT_INFRARED are set): the effective temperature (K) of the radiation in the evolved IR band.

RadiativeAcceleration: [N,3]-array (only included if RT_RAD_PRESSURE_OUTPUT is set): the acceleration from radiation pressure on each gas element, in physical code units (same units as HydroAcceleration).

Some quantities are specific to the star particles formed during the simulation (Type=4), including:

Metallicity (P[‘z’]): same as gas Metallicity array, but for stars. Stars inherit this from their progenitor gas particle. They do not modify it with time (even though stellar evolution is accounted for in the return of products to the ISM).

StellarFormationTime (P[‘age’]): [N]-array. The code “Time” when the star particle FORMED (not really an age). For cosmological runs this is the scale factor (\(a_{\rm form} = 1 / (1 + z_{\rm form})\)) at the redshift \(z_{\rm form}\) when the star particle formed. For non-cosmological runs it is the time (in \(h^{-1}\) Gyr) when the star particle formed.

IMFFormationProperties: [N,13]-array (only included if GALSF_SFR_IMF_VARIATION is set; Types 4 and 5): properties of the parent gas element at the moment the star formed, for use in IMF models. Columns, all in physical code units unless noted: 0 = the IMF turnover mass in \(M_{\odot}\), or with SINGLE_STAR_SINK_DYNAMICS the distance to the nearest sink; 1 = gas density; 2 = specific internal energy; 3 = effective sound speed; 4 = norm of the trace-free shear part of the velocity gradient; 5 = element size \(h\); 6 = local gas surface density (gradient estimator); 7 = total velocity dispersion across \(h\) (including Hubble flow); 8 = |gravitational acceleration|; 9 = |velocity|; 10 = |B| in Gauss; 11 = incident UV flux in Habing units (1 if GALSF_FB_FIRE_RT_LONGRANGE is off); 12 = cosmic-ray energy density (0 without COSMIC_RAY_FLUID).

OStarNumber: [N]-array (only included if GALSF_SFR_IMF_SAMPLING is set; Type 4): the integer number (stored as float) of massive (\(>8\,M_{\odot}\)) stars sampled for the particle, drawn from a Poisson distribution with mean \(0.0115\,m_{\ast}/M_{\odot}\). With GALSF_SFR_IMF_SAMPLING_DISTRIBUTE_SF it starts at 0 and grows as the massive stars are spawned.

TimeDistribOfStarFormation: [N]-array (only included if GALSF_SFR_IMF_SAMPLING_DISTRIBUTE_SF is set; Type 4): the time interval after formation over which the particle’s massive stars are spawned. It is the value of that flag times \(0.54/\sqrt{G\rho}\) of the parent gas (i.e. that many free-fall times), in physical code time units.

StellarKernelMaxRadius: [N]-array (only included if SUBFIND is set; Type 4): the kernel radius used by SUBFIND for its density estimate of the star particle (set by DesLinkNgb neighbors), in code length units. (This was StellarSmoothingLength in older snapshots; the code still reads that name on restart.)

DensityAtParticleLocation: [N]-array (only included if OUTPUT_DENS_AROUND_NONGAS is set; Types 1–5): the kernel-weighted gas density at the position of the non-gas particle, computed in the same density pass as for gas, in code units (comoving, like Density).

Some quantities are also specific to sink particle particles (Type=5), including:

Sink_Mass (P[‘mbh’]): [N]-array; the mass of the BH (not necessarily the same as the mass of the particle)

Sink_Mdot (P[‘mdot’]):[N]-array; the accretion rate of the BH evaluated at this instant (in code units MASS/TIME)

Sink_Mass_Reservoir: [N]-array (only included if SINK_ALPHADISK_ACCRETION is active); the mass in the sub-grid viscous accretion disk (legacy name BH_Mass_AlphaDisk).

Sink_InitialMass: [N]-array (if SINK_PARTICLES): the total particle mass of the sink when it formed (code units).

Sink_Specific_AngMom: [N,3]-array (only included if SINK_FOLLOW_ACCRETED_ANGMOM is set): the specific angular momentum vector of the material accreted by the sink, as a mass-weighted running average, in code units.

Sink_Radius: [N]-array (only included if SINK_GRAVCAPTURE_FIXEDSINKRADIUS is set): the fixed sink radius used for gravitational capture (code length units).

Sink_AccretionLength: [N]-array (only included if SINK_GRAVCAPTURE_FIXEDSINKRADIUS is set): the sink’s neighbor-search (kernel) radius used for its accretion and feedback interactions with gas (code length units).

Sink_Dust_Mass: [N]-array (if SINK_PARTICLES and GRAIN_FLUID): the cumulative mass of dust grain super-particles accreted by the sink (code units).

Sink_NProgenitors: [N]-array of integers (only included if SINK_COUNTPROGS is set): the number of sinks merged into this one, counting itself (1 for a sink that has never merged).

Sink_Distance: [N]-array (only included if SINK_CALC_DISTANCES and OUTPUT_SINK_DISTANCES are set; written for every particle type, not just sinks): the distance from the particle to the nearest sink particle, from the gravity tree walk, in code length units.

Unspawned_Wind_Mass: [N]-array (if SINK_WIND_SPAWN): mass already removed from the sink for winds or jets that has not yet been spawned as new gas cells (code units). This is evolving state, so it is always written when the flag is set, and it is read back on restart.

Unspawned_Jet_Mass: [N]-array (if SINK_WIND_SPAWN and SINGLE_STAR_FB_JETS): the separate reservoir of jet mass awaiting spawning (code units). Once the star reaches ProtoStellarStage 6, this reservoir is moved into Unspawned_Wind_Mass.

If SINGLE_STAR_STARFORGE_PROTOSTELLAR_EVOLUTION is set (in addition to ZAMS_Mass):

ProtoStellarStage: [N]-array of integers: the protostellar evolution stage (following Offner et al. 2009, Appendix B): 0 = pre-collapse, 1 = no burning, 2 = fixed-\(T_{c}\) D burning, 3 = variable-\(T_{c}\) D burning, 4 = shell D burning, 5 = main sequence, 6 = supernova.

ProtoStellarRadius_inSolar: [N]-array: the stellar (protostellar) radius in \(R_{\odot}\).

StarLuminosity_Solar: [N]-array: the total luminosity of the star in \(L_{\odot}\).

Mass_D: [N]-array: the mass of gas in the protostar that still contains unburned deuterium (code mass units; equal to the sink mass at formation).

ProtoStellarAge: [N]-array: the code Time at which the sink formed (same convention as StellarFormationTime: the scale factor in cosmological runs).

Some quantities are specific to dust-grain or MHD-PIC super-particles (the types in GRAIN_PTYPES, Type=3 by default):

GrainSize: [N]-array (if GRAIN_FLUID): the grain radius in cm. At startup it is sampled uniformly in log between Grain_Size_Min and Grain_Size_Max. It is 0 for PIC sub-types \(\geq3\) (non-dust species).

PICParticleType: [N]-array of integers (if PIC_MHD): the MHD-PIC sub-type MHD_PIC_SubType of each super-particle. Sub-types \(\leq2\) are dust species; \(\geq3\) are relativistic or cosmic-ray species.

Additional physics-dependent output fields (only present when the relevant compile-time flags are active):

Equation-of-State fields (when EOS_GENERAL, EOS_TILLOTSON, EOS_ANEOS, or EOS_HELMHOLTZ is active): Temperature (gas temperature in K), SoundSpeed (adiabatic sound speed), Pressure (gas pressure). When EOS_TILLOTSON or EOS_ANEOS is active: CompositionType (integer material index). When EOS_ANEOS is active: PhaseID (material phase flag: solid/liquid/vapor/mixed). When EOS_HELMHOLTZ is active: Ye (electron fraction), Abar (mean atomic weight).

StressTensor: [N,9]-array (only included if EOS_ELASTIC is set): the deviatoric elastic stress tensor \(S_{ab}\) of each element, in code pressure units (it is symmetric). Element [i, 3*a + b] is \(S_{ba}\). The isotropic pressure is in Pressure and is not included here.

If EOS_DAMAGE_POROSITY is set (with EOS_TILLOTSON/EOS_ELASTIC solids), three floating-point fields are written for all gas elements, whichever bits of the flag are on:

Damage: [N]-array: the Grady-Kipp scalar damage \(D\), between 0 (intact) and 1 (fully fractured). It never decreases. It is evolved only if bit 1 of EOS_DAMAGE_POROSITY is set.

ActiveCracks: [N]-array: the radius of the growing cracks in the Grady-Kipp model. It grows at \(0.4\,c_{s}\) and never decreases. Code length units. \(D = n_{\rm act}\,(4\pi/3)\,A^{3}\), with \(n_{\rm act}\) the Weibull active-flaw density.

Distention: [N]-array: the \(P\)-\(\alpha\) porosity distention \(\alpha = \rho_{\rm solid}/\rho \geq 1\) (1 = fully compacted). It starts at the material value \(\alpha_{0}\) and is evolved only if bit 4 of EOS_DAMAGE_POROSITY is set.

ElectronTemperature: [N]-array (only included if TWO_TEMPERATURE_PLASMA is set): the electron temperature \(T_{e}\) in K. It is derived from the evolved electron internal energy and can differ from the ion/gas Temperature.

NuclearComposition: [N, N_isotopes]-array (only included if NUCLEAR_NETWORK is set): the mass fractions of the isotopes evolved by the nuclear network. They are the network slots of the internal Metallicity array, in network order; for the default aprox13 network these are He4, C12, O16, Ne20, Mg24, Si28, S32, Ar36, Ca40, Ti44, Cr48, Fe52, Ni56.

Cosmic ray fields (when COSMIC_RAY_FLUID is active; gas only): CosmicRayEnergy ([N, N_CRbins]: CR energy in each bin, integrated over the element’s volume, code units). CosmicRayDiffusivity ([N, N_CRbins], only if CRFLUID_DIFFUSION_MODEL is set): the effective CR transport coefficient of each bin (diffusion plus streaming, which enter the same way), in physical code units of length${2}$/time. CosmicRayAlfvenEnergyPM ([N, 2*N_CRbins], only if CRFLUID_EVOLVE_SCATTERINGWAVES is set): the energy in gyro-resonant Alfvén waves propagating forward (+) and backward (-) along B. Element [i, N_CRbins*s + k] is bin k, with s = 0 for + and 1 for -; code energy units. CosmicRayMomentumDistSlope ([N, N_CRbins], only if CRFLUID_EVOLVE_SPECTRUM is set): the power-law slope \(\alpha\) of the CR momentum distribution within each bin (\(df/dp\propto p^{\alpha}\)), computed from the evolved number and energy in the bin (dimensionless).

Radiation fields (when RADTRANSFER or similar RT flags are active): PhotonEnergy (radiation energy per frequency bin), PhotonFluxDensity (radiation flux vector per bin, if OUTPUT_RT_RAD_FLUX), PhotonOpacity (opacity per bin, if OUTPUT_RT_RAD_OPACITY), EddingtonTensor, IRBand_Radiation_Temperature and Dust_Temperature (if RT_INFRARED), RadiativeAcceleration (if RT_RAD_PRESSURE_OUTPUT).

Sink/compact object fields (PartType=5, when sink physics enabled): Sink_Mass (sink mass), Sink_Mdot (accretion rate), Sink_InitialMass (mass at formation), Sink_Specific_AngMom (specific angular momentum of accreted material). When protostellar evolution is active: ProtoStellarRadius_inSolar, ProtoStellarStage, ProtoStellarAge, StarLuminosity_Solar, Mass_D, ZAMS_Mass.

All the above are only some examples of the different quantities saved, which will depend on the specific code modules and physics you have enabled, as well as custom input/output options you can set. Check your snapshots to see what is, in fact, being saved. If you aren’t sure what a given saved quantity actually represents, look it up in the source code.

10. Log Outputs & Diagnostic Files

Standard output & error

The code will write to standard error in general only when there is an error which is not handled by a “clean” exit/code termination within the code (so it you see a bunch of outputs to stderr, you should try to find out whats happening!). The code will output a very large amount of information to stdout. This is generally channeled through a single ‘lead’ process (“Task 0”), to avoid slowing down processes, but all processes are capable of writing to stdout. It is recommended that you route this to a named file so that you can track it. An example from a single timestep might look like:

Sync-Point 1136365, Time: 0.168474, Redshift: 4.93562, Systemstep: 1.64792e-07, Dloga: 9.78146e-07  
Occupied timebins: non-cells     cells       dt                 cumulative A D    avg-time  cpu-frac  
    bin=28         6761427      974907     0.002003243831         11683119   *       19.57     14.8%  
    bin=27          474946     2175185     0.001001621915          3946785   *       12.45      9.4%  
    bin=26           86245      739536     0.000500810958          1296654   *        4.56      6.9%  
    bin=25           26163      238404     0.000250405479           470873   *        2.54      7.7%  
    bin=24           10514      114898     0.000125202739           206306   *        1.85     11.2%  
    bin=23            3232       54415     0.000062601370            80894   *        1.72     20.8%  
    bin=22               7       17988     0.000031300685            23247            0.15      3.5%  
    bin=21               5        3672     0.000015650342             5252            0.07      3.1%  
    bin=20               4         870     0.000007825171             1575            0.04      3.8%  
    bin=19               3         400     0.000003912586              701            0.03      6.0%  
    bin=18               0         248     0.000001956293              298            0.02      6.6%  
 X  bin=17               0          50     0.000000978146               50 <          0.01      6.2%  
               ------------------------  
Total active:            0          50    Sum:         50

Kick-subroutine will prepare for dynamic update of tree  
 ..exchanged kick momenta for 138 top-level nodes out of 234697  
 ..Tree has been updated dynamically  
Start gravity force computation...  
 ..Begin tree force. (presently allocated=2687.14 MB)  
 ..gravity force computation done  
Start hydrodynamics computation...  
 ..Hmax exchange: 13 topleaves out of 234697  
 ..density & tree-update computation done...  
 ..gradient computation done.  
 ..hydro force computation done.  
Start mechanical feedback computation...  
 ..mechanical feedback loop: iteration -2  
 ..mechanical feedback loop: iteration -1  
 ..mechanical feedback loop: iteration 0  
 ..mechanical feedback loop: iteration 1  
Local HII-Region photo-heating/ionization calculation  
Local Radiation-Pressure acceleration calculation  
 ..completed local Radiation-Pressure acceleration  
Sink operations begin...  
 ..closing sink operations  
Cooling and Chemistry update  
Predicting next timestep: 9.78146e-07

The top section shows the distribution of particles in different timebins (as in timebin.txt discussed below), then the file notes as it goes through different stages of calculation (updating the tree, walking the tree to calculate gravitational forces, entering the density computation, hydrodynamic gradients calculation, hydrodynamic forces, sink particle-specific calculations, calculating the next timestep and checking if particles need to be activated, etc). You should always monitor this output. This will be written regularly regardless of the output options. However the amount of output will be much more extensive if the compile-time flag OUTPUT_ADDITIONAL_RUNINFO is enabled (the example above included this). The extra output will be things like entering/exiting of different sub-routines, most of which is not important information (but can be useful for debugging).

info.txt (only output if OUTPUT_ADDITIONAL_RUNINFO is enabled)

In the file ‘info.txt’, you just find a list of all the timesteps. Typical output in this file looks like this:

Begin Step 49, Time: 0.143555, Systemstep: 0.00292969  
Begin Step 50, Time: 0.146484, Systemstep: 0.00292969  
Begin Step 51, Time: 0.149414, Systemstep: 0.00292969  
Begin Step 52, Time: 0.152344, Systemstep: 0.00292969

The first number just counts and identifies all the timesteps, the values given after ‘Time’ are their current simulation times, and the ‘Systemstep’-values give the time difference to the pre- ceeding step, which is equal to by how much the simulation as a whole has been advanced since the last step. For cosmological integrations, this output is supplemented with the redshift, and the systemstep in linear and logarithmic form:

Begin Step 636, Time: 0.487717, Redshift: 1.05037, Systemstep: 0.000724494, Dloga: 0.00148658  
Begin Step 637, Time: 0.488443, Redshift: 1.04732, Systemstep: 0.000725571, Dloga: 0.00148658  
Begin Step 638, Time: 0.489169, Redshift: 1.04428, Systemstep: 0.000726651, Dloga: 0.00148658  
Begin Step 639, Time: 0.489897, Redshift: 1.04125, Systemstep: 0.000727732, Dloga: 0.00148658  
Begin Step 640, Time: 0.490626, Redshift: 1.03821, Systemstep: 0.000728814, Dloga: 0.00148658

timings.txt (only output if OUTPUT_ADDITIONAL_RUNINFO is enabled)

In the file ‘timings.txt’, you get some statistics about the performance of the gravitational force computation by the tree algorithm, which is usually (but not always) the main sink of computational time in a simulation. A typical output may look like this:

Step= 482  t= 1.29199  dt= 0.00292969  
Nf= 10992  total-Nf= 0016981370  
work-load balance: 1.10997  max=0.618782 avg=0.557475 PE0=0.618782  
particle-load balance: 1.02133  
max. nodes: 15202, filled: 0.422278  
part/sec=9858.74 | 8881.96  ia/part=1004.71 (0)

The first line of the block generated for each step informs you about the number of the current timestep, the current simulation time, and the system timestep itself (i.e. the time difference to the last force computation). Nf gives the number of particles that receive a force computation in the current timestep, while total-Nf gives the total number of force computations since the simulation was started. The work-load balance gives the work-load balance in the actual tree walk, i.e. the largest required time of all the processors divided by the average time. The number given by max is the maximum time (in seconds) among the processors spent for the tree walk, while avg gives the average, and PE0 the time for processor 0. particle-load balance gives the maximum number of particles per processor divided by the average number, i.e. it measures the memory imbalance. An upper bound for this number is PartAllocFactor. However, for technical reasons, the domain decomposition always leaves room for a few particles, hence this upper bound is never exactly reached. max.nodes informs about the maximum number of internal tree nodes that are used among the group of processors, and the number behind filled gives this quantity normalised to the total number of allocated tree nodes, it hence gives the fraction to which the tree storage is filled. Finally, part/sec measures the raw force speed in terms of tree-force computations per processor per second. The first num- ber gives the speed that would be achieved for perfect work-load balance. Due to work-load imbalance, the actually achieved average effective force speed will always be lower in practice, and this number is given after the vertical line. ia/part gives the average number of particle-node interactions required to compute the force for each of the active particles. The number in parenthesis is only non-zero if the Ewald correction is used to obtain periodic boundaries. It then gives the average number of Ewald correction-forces evaluated per particle.

cpu.txt

In the file ‘cpu.txt’, you get some statistics about the total CPU consumption measured in various parts of the code while it is running. A typical output looks like this:

Step 1146466, Time: 0.169055, CPUs: 224  
Nactive=18, Imbal(Max/Mean)=1.17634  
total           44927.08  100.0%  
tree+gravity     6913.13   15.4%  
   treebuild     1493.23    3.3%  
   treewalk      1538.67    3.4%  
   treecomm       534.70    1.2%  
   treeimbal     2858.32    6.4%  
pm-gravity        558.37    1.2%  
hydro/fluids    13765.43   30.6%  
   dens+grad      965.28    2.1%  
   denscomm      1713.23    3.8%  
   densimbal     2471.50    5.5%  
   hydrofrc      1872.59    4.2%  
   hydcomm       2106.64    4.7%  
   hydimbal      3287.75    7.3%  
   hmaxupdate     105.58    0.2%  
   hydmisc       1242.87    2.8%  
domain           5736.25   12.8%  
peano             327.15    0.7%  
drift/splitmg    1641.10    3.7%  
kicks             268.89    0.6%  
io/snapshots       49.81    0.1%  
cooling+chem     6706.17   14.9%  
sinks       1412.80    3.1%  
mech_fb_loop     2656.93    5.9%  
hII_fb_loop      1946.22    4.3%  
localwindkik      204.51    0.5%  
misc             2740.33    6.1%

For each timestep, there is a large “block” printed out (actually this is only output on every top-level-timestep unless the OUTPUT_ADDITIONAL_RUNINFO flag is enabled). The top of the block specifies the current timestep number, simulation time, and number of CPUs used. Then for each item in the block, there is a name (specifying the task) and two numbers. The first number is the cumulative CPU consumption (per MPI task) for that task of the code up to this point (wallclock times in seconds), and the second is the percent of the total CPU time this corresponds to.

The blocks shown include:

total: Total consumption

tree+gravity: Gravitational force tree computation. This larger category is divided into various sub-categories, including tree building/construction and update/drifting (treebuild), walking to find neighbors/do the interactions (treewalk), communication (treecomm), and the imbalance (treeimbal) calculated as the time wasted while CPUs wait for other CPUs to finish their tree operations before the code can move forward.

hydro/fluids: Hydrodynamics and all other related fluid dynamics, elastics, radiation-hydrodynamics, etc, calculations. This has two subsets: the “density” section includes the initial density calculation, kernel length determination, and gradients calculations. This is divided into the computation (dens+grad), communication (denscomm), and imbalance (densimbal). The “hydro” section includes the calculation of hydrodynamic forces and all other work in the hydro loop, such as radiation transfer between cells (hydrofrc), communication (hydcomm), imbalance (hydimbal), miscellaneous (hydmisc), and network costs (for nuclear networks). The value hmaxupdate traces the cost of updating kernel lengths in the tree from the density computations.

domain: Domain decomposition.

peano: Time needed to establish Peano-Hilbert order.

drift/splitmg: Drifting the particle set and performing cell merge/split operations.

kicks: Timestep determination and ‘kicking’ of particles.

io/snapshots: Writing of snapshot files.

misc: Anything that doesnt fit in the above.

Various optional physics modules will add their own blocks to the table here, so you can diagnose how expensive the routines are. Some examples are:

pm-gravity: Gravitational force computation from the PM grid on large scales (when hybrid Tree-PM mode is used).

potentialeval: Potential energy computation (for the energy statistics). If any flags are on which require “normal” evaluation of the potential throughout the run, then this is not included, because this potential calculation is done directly alongside gravity (costs included above). But if OUTPUT_POTENTIAL is enabled and this is not done otherwise a special call must be made before snapshots to compute this, leading to this entry appearing.

ags-nongas: Routines which require kernel density calculations for non-fluid elements (e.g. adaptive gravitational softening for stars, self-interacting dark matter, etc) must evaluate a density-like loop, whose costs are summed here, broken up like hydro costs into the main density/kernel loop (agsdensity), communications (agscomm), imbalance (agsimbal), and miscellaneous (agsmisc).

dyndiff and velsmooth: Dynamic diffusion (TURB_DIFF_DYNAMIC) flags will activate these, with the multi-kernel dynamic diffusion tensor calculation and smoothed velocity field construction broken into compute costs (compute), communication (comm), imbalances (wait), and miscellaneous (misc)

fof/subfind: Cost for on-the-fly friends-of-friends structure or substructure finding (if e.g. FOF or SUBFIND is active).

cooling+chem: Cooling routines and astro-chemical libraries (COOLING active). If CHIMES is active, an additional diagnostic of imbalance (coolchmimbal) is added since the large network solve can lead to large imbalances this can help diagnose.

sinks: sink particle physics (neighbor finding, accretion, and feedback routines), if SINK_PARTICLES active.

grains: Grain/dust/cosmic ray MHD-PIC dynamics (Lorentz force, drag, collisions, and other calculations specific to these flags).

sfrcool: Time spent in cooling and star formation routines.

mech_fb_loop: Mechanical feedback routines (e.g. supernovae, stellar mass-loss, from stellar sources, if GALSF_FB_MECHANICAL or GALSF_FB_THERMAL is active).

hII_fb_loop: FIRE HII photo-ionization routines (GALSF_FB_FIRE_RT_HIIHEATING).

localwindkik: FIRE local radiation pressure calculation (GALSF_FB_FIRE_RT_LOCALRP).

rt_nonfluxops: Collects radiation-hydrodynamics operations (with RADTRANSFER active) not included already in the hydrodynamics solvers total above (those contain the cell-cell fluxes) and basic drift/kick operations (also already included above).

energy.txt (only output if OUTPUT_ADDITIONAL_RUNINFO is enabled)

In the file ‘energy.txt’, the code gives some statistics about the total energy of the system. In regular intervals (specified by TimeBetStatistics), the code computes the total kinetic, thermal and (optionally) potential energy of the system, and it then adds one line to EnergyFile. Each of these lines contains 28 numbers, which you may process by some analysis script. The first number in the line is the output time, followed by the total internal energy of the system (will be 0 if no gas physics is included), the total potential energy, and the total kinetic energy. The next 18 numbers are the internal energy, potential energy, and kinetic energy of the six particle types Gas to Bndry. Finally, the last six numbers give the total mass in these components.

While the meaning of kinetic and potential energy in non-cosmological simulations is straightforward (and also the test for conservation of total energy), this is more problematic in cosmological integrations. See the GADGET user’s guide for details (section 7.4). The output kinetic energy is the sum of \(0.5\,m_{i}\,v_{i}^{2}\) where \(v_{i} = w/\sqrt{a}\) is the peculiar velocity divided by \(\sqrt{a}\). The potential energy is the energy from the peculiar potential. While checking energy conservation is straightforward for Newtonian dynamics, it is more difficult for cosmological integrations, where the Layzer-Irvine equation is needed. (Note that the softening needs to be fixed in comoving coordinates for it to even be defined, which is not true when adaptive softenings are used.) It is in practice not easy to obtain a precise value of the peculiar potential energy at high redshift. Also, the cosmic energy integration is a differential equation, so a test of conservation of energy in an expanding cosmos is less straightforward that one may think, to the point that it is nearly useless for testing code accuracy.

timebin.txt (only output if OUTPUT_ADDITIONAL_RUNINFO is enabled)

The file ‘timebin.txt’ shows the distribution of particles in different adaptive timesteps. A typical output looks like this:

Sync-Point 436423, Time: 0.999991, Redshift: 8.80268e-06, Systemstep: 4.40127e-06, Dloga: 4.40132e-06  
Occupied timebins: non-cells     cells       dt                 cumulative A D    avg-time  cpu-frac  
    bin=18         2182704     1192362     0.002253476820          6350412   *       11.89      2.4%  
    bin=17         1016844       36194     0.001126738410          2975346   *        6.21      1.3%  
    bin=16          926238       15270     0.000563369205          1922308   *        5.61      2.3%  
    bin=15          532179       13099     0.000281684602           980800   *        5.00      4.1%  
    bin=14          148120       21879     0.000140842301           435522   *        4.26      6.9%  
    bin=13           70947       23283     0.000070421151           265523   *        3.94     12.8%  
    bin=12           59074       11274     0.000035210575           171293   *        3.29     21.5%  
    bin=11           91438        1967     0.000017605288           100945   *        2.76     36.0%  
 X  bin=10            4908         353     0.000008802644             7540 <          0.24      6.2%  
 X  bin= 9            2228          51     0.000004401322             2279            0.12      6.5%  
               ------------------------  
Total active:         7136         404    Sum:       7540

The top line shows the current timestep number (sync-point), time, corresponding redshift (if cosmological), systemstep (minimum timestep), and corresponding dloga (for cosmological integrations). The table below shows each timebin with particles in it – these are the hierarchical powers-of-two timestep levels (higher bins are particles with larger timesteps, and each bin has a timestep =0.5 that above it). The first column (“X”) shows which bins are active that timestep, then we have the bin number, the number of non-gas particles (“non-cells”) per bin, number of gas particles (“cells”), timestep (“dt”), cumulative number of particles (all types) in bins <= the bin, the currently largest active bin (header “A” and bin denoted with the “<”), the bins which require a new domain decomposition (every bin with a “\(*\)” in the “D” column requires domain decomposition: this is ultimately determined by the parameter DomainBuild_ActiveFraction in the parameterfile), the average time (wall-clock time) per cycle per MPI process currently estimated for the timebin, and the on-the-fly estimated fraction of the total CPU time used for each bin (this information is used to try to build a balanced domain decomposition in the code).

balance.txt

This is a rather technical and hard-to-parse file containing the balance information for each processor at each timestep. Consult the GADGET users guide if you need it, but you usually shouldnt. The opening of the file contains a key which labels each of the different categories of CPU flags, and details over each timestep the fraction of time spent in each as the code works its way through. It is more detailed than what you get from cpu.txt, but less useful often than what you would get from using code profiling tools (which is recommended for real optimization, this is more of a useful tool for debugging). In newer versions of GIZMO, this file is not output by default unless additional outputs are enabled.

Outputs specific to different physics modules:

Various specific physics modules (enabled by different compile-time flags) will output their own files. Some examples are given below:

sfr.txt

Models with star formation (GALSF on) will output this file. An output looks like: 

0.999991 4.72152e-06 2.92239 1.09686 0  
0.999996 7.30842e-06 3.70939 1.69782 0

Columns are: (1) simulation time, (2) expectation value of mass in stars the SF should form that timestep, (3) total SFR in code units, (4) discretized total stellar mass/timestep in solar masses per year, (5) total mass in stars actually spawned (new particles) that timestep. Unless OUTPUT_ADDITIONAL_RUNINFO is enabled, this is only output on top-level or domain-level timesteps.

HIIheating.txt

Models with local photoionization heating (GALSF_FB_FIRE_RT_HIIHEATING) will write ‘HIIheating.txt’

0.0442866 1 3.04398e+52 1 267105 1.25796

Columns are (1) simulation time, (2) number of sources emitting ionizing photons (very young stars active that timestep), (3) total ionizing photon emission in photons/s, (4) total number of gas cells ionized (all or in part) by those stars, (5) total gas mass (in solar) ionized, (6) average size of the HII regions being created (distance from source to ionized gas element, in code units)

MomWinds.txt

Models with the short-range local radiation pressure (GALSF_FB_FIRE_RT_LOCALRP) will write ‘MomWinds.txt’

0.256422469 1478 96 18173 2.48201 0.0134358 0.00263988 0.0298646 4.49967e-06

Columns are (1) simulation time, (2) number of star particles active this timestep and checked against the emission criterion, (3) number of those which actually emit, after the stochastic criterion which lets the routine fire rarely and couple a correspondingly larger momentum when it does (so this is typically a small fraction of the previous column), (4) total number of gas cells which receive momentum from those stars, (5) mean radius searched around each emitting star (code units), which is where the cost of the routine comes from, since the work scales with the number of cells inside it, (6) total photon momentum (“\(dt\,L / c\)”) of the emitting stars for which the radiation pressure is being calculated (code units) for the timestep, (7) total momentum actually coupled to gas – note this can and usually does -exceed- the previous column, because the infrared term couples the same photon momentum repeatedly where the gas is optically thick to its own re-radiation, (8) average velocity of the discretized “kicks” assigned to gas particles (acceleration \(*\) dt) from the momentum, (9) mean infrared multiple-scattering boost -per coupled cell-: for each cell, the infrared kick it is offered against the bare single-scattering momentum per unit mass, averaged over cells and weighted by the momentum each one receives. This last is a per-cell mean and not the total boost to the coupled momentum in column (7); the momentum weighting pulls it toward the cells nearest the source, so it sits well above the median cell and should not be read as a characteristic optical depth.

Note that a line is written on every timestep, including those on which stars were active but none of them passed the emission criterion (columns 3–9 are then zero) – the comparison of columns (2) and (3) is the diagnostic for whether that criterion is behaving as intended.

MechFeedbackEvents.txt

Models with mechanical feedback from SNe and stellar winds (GALSF_FB_MECHANICAL) will write ‘MechFeedbackEvents.txt’

0.00482178 2546 9 9 7.00321 5.71995e-05 1.74019e-06

Columns are (1) simulation time, (2) number of (star) particles active which could (potentially) have SNe (or stellar wind mass ejection event), (3) total number of particles which actually have at least one SNe (or event), (4) total number of SNe (or events) this timestep (this can be larger than the number of host stars if the timestep is long enough), (5) exact (non-integer) integral of the rate functions over the timestep for all events, which should equal the expectation value of the number of discrete events, given in the previous column, (6) mean timestep of the active star particles (dt), (7), mean SNe rate of the active star particles at this time.

sinks.txt

Models with black holes or stars or sink particles active (SINK_PARTICLES) will write ‘sinks.txt’

0.16714 33 0.000221363 0.000365694 0.00373909 0.00103651 0.336703

Columns are (1) simulation time, (2) total number of sink particles/sinks (BHs) in the simulation, (3) total mass (in code units) of all BHs (note this uses the Sink_Mass flag, not the total dynamical mass or the reservoir mass for BH particles, but the mass nominally fully associated with the sink/BH itself), (4 ) total accretion rate (code units) summed over all BHs (again, this uses the accretion rate onto the BHs themselves, if the alpha-disk or reservoir model is turned on, not the accretion rate into said reservoir, which can be a series of delta functions depending on the accretion model employed), (5) total accretion rate in solar masses per year, (6) total of all the dynamical masses of all BH particles (i.e. including the full particle dynamical mass, plus reservoir, plus BH, which can in principle be larger than the BH by a large margin), (7) mean Eddington ratio \(\langle \dot{M}_{\rm BH} / \dot{M}_{\rm Edd}(M_{\rm BH}) \rangle\) of all the BHs (note that the mean weighted by mass, or \(\langle \dot{M}_{\rm BH} \rangle / \langle \dot{M}_{\rm Edd}(M_{\rm BH}) \rangle\) can be directly calculated from the information in the other columns here).

11. Test Problems

Here we outline the setup, initial conditions files, and parameterfiles needed to run a large number of interesting code test problems. Most of these problems come from the code paper; we refer you to that paper for detailed discussion of each test as well as comparisons between various numerical methods.

Running Tests with pytest

GIZMO includes a pytest-based test suite that automates building, running, and validating test problems. To use it:

Step 1: Install GIZMO as a Python package. From the GIZMO source directory, run pip install . (or pip install -e . for an editable/development install). This installs the gizmo Python package, which includes test utilities and snapshot-reading tools. Dependencies include pytest, h5py, numpy, scipy, matplotlib, astropy, and meshoid.

Step 2: Ensure your build environment works. Make sure Makefile.systype has the correct system type uncommented for your machine (e.g. SYSTYPE=MacBookCellar for a typical macOS homebrew setup).

Step 3: Run tests. From the GIZMO source directory:

# Run all tests  
pytest

# Run a specific test  
pytest test/shocktube

# Run several tests by pattern  
pytest test/sedov test/noh test/shocktube

# Verbose output  
pytest test/sedov -v

# Show stdout during the test (useful for debugging)  
pytest test/sedov -v -s

Each test automatically: (1) builds GIZMO with the test-specific Config.sh flags, (2) downloads initial conditions and reference solutions if needed, (3) runs the simulation with appropriate MPI/OpenMP parallelization, and (4) validates results against analytical solutions or reference data. By default, tests use half the available CPU cores as MPI ranks and 2 OpenMP threads per rank.

Interpreting results. A passing test prints PASSED; a failing test shows the assertion that failed (e.g. “Density L1 error 0.12 exceeds tolerance”). If a test fails, check test/{test_name}/test_{test_name}.err for compiler or runtime errors, and test/{test_name}/test_{test_name}.out for GIZMO stdout. Simulation output snapshots are in test/{test_name}/output/, and diagnostic plots (if generated) are in test/{test_name}/.

Adding new tests. See test/how_to_add_new_tests.md for instructions. Each test lives in test/{test_name}/ and requires: a pytest file test_{test_name}.py, a Config.sh with compile flags, a {test_name}.params parameterfile, and optionally a README.md describing the test physics.

Test Problem Descriptions

All the ICs and example parameterfiles mentioned here – enough to let you run every one of these tests, is available in the directory here:

http://www.tapir.caltech.edu/~phopkins/sims/

The names should be fairly self-explanatory, but are given in the descriptions of each problem, just in case.

These are intended to be instructive, NOT the final answer for any of these problems. The values in the parameterfiles are basically copy-pasted from whatever I was testing at the time this guide was written, they are by no means a recommendation as the “best” parameters for any of the tests. The whole point of test problems is that you should experiment to see how the results depend on these choices, and what they are (or are not) sensitive to. For the same reason, these aren’t exactly the same choices used in the runs shown in the code paper; moreover, because codes are always being edited and improved, your version of GIZMO may not match the earliest version used in that paper. So you shouldnt expect to get exactly the same answer for your tests as is shown there (though if you are running in the same mode with similar parameters, they should be pretty close).

Similarly, the ICs here are intended to get you started - you could obviously choose to run the problems with different resolutions or different initial setups. If you need the routines used to generate a particular IC, I can provide some of them (some of the ICs are themselves taken from other sources). However I encourage you, when possible, to generate your own, since it will give you a much better understanding of how these problems are set up.

I’ve also made an effort to “pre-package” these as much as possible, so you can run them with relatively little effort. They don’t come with any guarantees though!

Equilibrium and Steady-State Tests

Linear Traveling Soundwave

This is a simple linear one-dimensional traveling soundwave, following the example here

This is problem is analytically trivial; however, since virtually all schemes are first-order for discontinuities such as shocks, smooth linear problems with known analytic solutions are the only way to measure and quantitatively test the accuracy and formal convergence rate of numerical algorithms. We initialize a periodic domain of unit length, with a polytropic \(\gamma=5/3\) gas with unit mean density and internal energy set such that the sound speed \(c_s=2/3\). We then add to this a traveling soundwave with small amplitude \(\delta \rho/\rho = 10^{-6}\) (to avoid any non-linear effects) with unit wavelength. After the wave has propagated one wavelength (\(t=1.5\)), it should have returned exactly to its initial condition.

Initial conditions are soundwave_ics.hdf5

An example run-time parameterfile is “soundwave.params” (pytest: test/soundwave)

Note that most of the run-time parameter settings in the file are totally irrelevant to the problem (things like gravitational softenings, etc); they are just here for completeness. And quantities like the memory allocation, cpu run time, etc should be modified appropriately for the machine you’re using. Also, all the quantities governing integration accuracy can be adjusted, you should experiment to see how this alters the results. Since this is designed for convergence studies, the tolerances here are set fairly ‘strictly’.

Make sure the following flags are set in the Config.sh file:

BOX_PERIODIC  
BOX_SPATIAL_DIMENSION=1  
SELFGRAVITY_OFF  
OUTPUT_IN_DOUBLEPRECISION  
INPUT_IN_DOUBLEPRECISION  
EOS_GAMMA=(5.0/3.0)

Plus a flag to specify the hydro solver you want to use, e.g. HYDRO_MESHLESS_FINITE_MASS (MFM), HYDRO_MESHLESS_FINITE_VOLUME (MFV), or an SPH variant. All test problems listed below similarly require a hydro solver flag; the automated tests in the test/ directory each include the solver in their Config.sh (typically HYDRO_MESHLESS_FINITE_MASS).

The Square Test

Here we test how well a contact discontinuity can be advected, particularly one with a non-trivial boundary shape. This is a major problem in grid/AMR codes, where there is far too much diffusion. While much improved, it is also surprisingly difficult to maintain a ‘clean’ edge of the square in SPH, because of spurious surface tension and pressure gradient errors along the discontinuity.

We initialize a two-dimensional fluid in a periodic box of length \(L=1\) and uniform pressure \(P=2.5\), \(\gamma=1.4\), and density \(\rho=4\) within a central square of side-length \(L=1/2\) and \(\rho=1\) outside. The entire fluid is given a uniform and abitrary large initial velocity \(v_{x} = 142.3\), \(v_{y} = -31.4\). We use \(64^{2}\) particles. The square should be in hydrostatic equilibrium, with the final state exactly identical to the initial state (after re-centering on the square). In the paper, we evolve this to \(t=10\).

Initial conditions are square_ics.hdf5

Parameterfile is “square.params” (pytest: test/square)

In Config.sh, enable:

BOX_PERIODIC  
BOX_SPATIAL_DIMENSION=2  
SELFGRAVITY_OFF  
EOS_GAMMA=1.4

Plus your hydro solver!

The Gresho Vortex

This is the triangular vortex of Gresho and Chan (1980). A two-dimensional gas with uniform \(\rho=1\) is initialized in a periodic domain \(0 < x < 1\), \(0 < y < 1\), with zero radial velocity, and a pressure gradient designed to exactly balance rotational motion (see the code paper for the details of the setup). The problem tests the ability of a code to handle sub-sonic turbulence and control very small pressure gradient errors.

The system should represent a steady-state equilibrium vortex (the exact solution for the velocity, pressure, and density fields should be equal to the initial condition). SPH has serious difficulties with this test, as do non-moving grid codes if the vortex has a bulk motion relative to the grid (which is almost always the case in any real problem).

Initial conditions: gresho_ics.hdf5 or gresho_ics_grid.hdf5

Parameterfile: “gresho.params” (pytest: test/gresho)

In Config.sh, enable:

BOX_PERIODIC  
BOX_SPATIAL_DIMENSION=2  
SELFGRAVITY_OFF  
EOS_GAMMA=(1.4)

In the paper, we evolve this to \(t=3\). Many SPH studies only evolve to \(t=1\), because this is a particularly challenging problem for SPH (you will find you need to increase the neighbor number substantially to get anything decent looking with SPH methods). If you want to run a grid-code comparison, you should add a constant velocity to the vortex: this wont have any effect on the GIZMO hydro methods, but will substantially change the grid code result (the result with no boost looks prettier, but it is fairly misleading about the accuracy of convergence of the code).

This problem is highly sensitive to the slope-limiters used at this resolution (in any code), because of the sharp but un-resolved peak in the velocity distribution at the center of the vortex. So experiment with the different SLOPE_LIMITER_TOLERANCE settings. Likewise the different kernel functions set by KERNEL_FUNCTION can make a substantial difference (try setting to 5 or 7, with 40 neighbors, instead of the default 3, with 20 neighbors). Similarly experiment with the hydro solver options like HYDRO_REPLACE_RIEMANN_KT and KERNEL_CRK_FACES.

Note that the IC file here is simple particles on concentric rings around the center, which is not actually the most accurate setup because the shear immediately distorts these (a spiral pattern or glass gives better results). It is not a useable IC if you want to run GIZMO in, say, fixed regular-mesh mode – we therefore provide the alternative gresho_ics_grid.hdf5 file which puts the ICs on a regular grid. This allows you to compare to the methods listed as “MFV-E” for example in the GIZMO methods paper.

​

Keplerian Disks

We now consider a cold Keplerian disk test problem. This is a critical problem for understanding the ability of codes to conserve angular momentum and follow gas in gravitational orbits for any length of time. Disks, in particular, are ubiquitous in astrophysics, and unlike the vortex problem above are dominated by gravitational acceleration rather than pressure forces (with the rotation velocity often super or trans-sonic); this focuses on that regime.

We initialize a razor thin \(\gamma=5/3\) disk with constant surface density \(\Sigma={\rm d}M/{\rm d}A = 1\), vanishingly small pressure, and all gas on circular orbits about the origin subject to a Keplerian external acceleration \(\ddot{{\bf r}} = - \hat{r}\,G\,M_{0}/|{\bf r}|^{2}\) with \(G=M_{0}=1\) (and no self-gravity) and \({\bf r}\) the vector from the origin. As in the above tests, the exact solution should simply be identical to the initial condition.

Cartesian mesh (grid/AMR) codes have serious problems with this test, and the disk is usually destroyed in a couple orbits. For simplicity, the test here is just the two-dimensional case; if you try to run a 3D case in a grid code and the disk is not perfectly aligned with the grid axis, the problems will be much worse still!

SPH does much better, if you use a more sophisticated artificial viscosity scheme; otherwise the artificial viscosity breaks up the disk nearly as badly as in the grid-code case within a couple orbits. However SPH doesn’t suffer from any of the 3D “grid alignment” errors that plague grid codes.

The MFM and MFV methods, on the other hand, should perform vastly better on this problem than AMR or SPH.

Initial conditions: keplerian_ics.hdf5

Parameterfile: “keplerian.params” (pytest: test/keplerian)

In Config.sh, enable:

BOX_SPATIAL_DIMENSION=2  
SELFGRAVITY_OFF  
GRAVITY_ANALYTIC  
EOS_GAMMA=(5.0/3.0)

And make sure you go into the file gravity/analytic_gravity.h and un-comment the line:

//GravAccel_KeplerianTestProblem();   // keplerian disk with boundaries for test problem

within the routine add_analytic_gravitational_forces. Otherwise the code won’t know what to use for the analytic gravitational forces.

Shock and Non-Linear Jump Tests

Sod Shock Tube

This is a standard Riemann problem (similar to those used in many basic code tests). We set up a one-dimensional Sod shock tube with a left state (\(x < 0\)) described by \(P_{1}=1\), \(\rho_{1}=1\), \(v_{1}=0\) and right state (\(x\ge 0\)) with \(P_{2}=0.1795\), \(\rho_{2}=0.25\), \(v_{2}=0\), and \(\gamma=1.4\). These parameters follow e.g. Hernquist and Katz 1989, Rasio and Shapiro 1991, Wadsley 2004, Springel 2005, and others. The test is modest resolution, with an initial \(100\) particles in the range \(-10 < x < 10\) (spacing \(\Delta x \approx 0.2\)). Note that the box is more extended, but the rest of it is ‘filler’.

Initial conditions: shocktube_ics_emass.hdf5 and shocktube_ics_diffmass.hdf5

Note there are two different initial conditions, for comparison here: one (shocktube_ics_emass.hdf5) with equal-mass particles (but variable particle spacing) and one (shocktube_ics_diffmass.hdf5) with discontinuous particle masses but equal particle spacing. This lets you explore how those different choices make a difference for the distinct numerical methods.

Parameterfile: “shocktube.params” (pytest: test/shocktube)

As in all the tests above most of the parameters here are irrelevant, but this is included just to get you started

In Config.sh, enable:

BOX_PERIODIC  
BOX_SPATIAL_DIMENSION=1  
SELFGRAVITY_OFF  
OUTPUT_IN_DOUBLEPRECISION  
INPUT_IN_DOUBLEPRECISION  
EOS_GAMMA=1.4

Unlike the ‘steady-state’ problems, the exact answer here is not the same as the initial conditions. If you go to the same website with the initial conditions and parameterfile, and get the file shocktube_exact.txt you’ll have an ASCII table with the exact solution for the shocktube at time \(t=5.0\). That file explains the different columns to which we compare in the code paper (density, pressure, velocity, and entropy, as a function of x-position; entropy is defined by the entropic function \(s=P/\rho^{\gamma}\) ). If you vary slope limiters, you can see how this affects the ‘ringing’ of solutions. But all the methods should do fairly well on this test, with subtle differences.

Interacting Blastwaves (Woodward and Colella)

Another related one-dimensional test problem is the interaction of two strong blast waves, from Woodward and Colella 1984. We initialize gas with density \(\rho=1\), \(\gamma=1.4\), \(v=0\) in the domain \(0< x <1\), with pressure \(P=1000\) for \(x<0.1\), \(P=100\) for \(x>0.9\), and \(P=0.01\) elsewhere. The boundary conditions are reflective at \(x=0\) and \(x=1\). This features multiple interactions of strong shocks and rarefactions, and provides a powerful test of Reimann solvers (many crash on this test, and essentially all codes require some hierarchy of solvers as we have implemented). We use \(400\) particles initially evenly spaced and equal-mass. In this problem, fixed-grid codes – even without bulk advection of the problem – will substantially diffuse the shock jumps because contact discontinuities must be advected. SPH does quite well, actually. MFM and MFV methods do well, but some care is needed to prevent the Reimann solver crashing (true of any Reimann-solver based method).

Initial conditions: interactblast_ics.hdf5

Parameterfile: “interactblast.params” (pytest: test/interactblast)

In Config.sh, enable:

BOX_BND_PARTICLES  
BOX_REFLECT_X  
BOX_SPATIAL_DIMENSION=1  
SELFGRAVITY_OFF  
OUTPUT_IN_DOUBLEPRECISION  
INPUT_IN_DOUBLEPRECISION  
EOS_GAMMA=1.4

The file interactblast_exact.txt has the exact solution at time \(t=0.038\). Most versions of the test focus on the comparison of the density as a function of position. In fixed-grid codes, diffusion associated with advecting the contact discontinuities smears out many of the density jumps. SPH should do well. If you experiment with things like neighbor number and condition number criteria for gradient matrices, you may find some of the Riemann-based methods crash, in the extreme shocks presented by this problem. It is also quite sensitive to the choice of time-stepping – too large a timestep will lead to too big an ‘initial’ shock propagation, which will lead to slightly offset shock positions.

Sedov Blastwaves / Point Explosions

Here we consider a Sedov-Taylor blastwave, a point explosion with large Mach number. This provides a powerful test of the accuracy of code conservation, as well as of how well codes capture shock jumps and preserves symmetry in three dimensions. When adaptive (non-constant) timesteps are used (as they are in our code) this is also an important test of the integration stability of the method (see Saitoh and Makino 2009).

This particular Sedov-Taylor test is from the Wengen test suite. A box of side-length 6 kpc is initially filled with \(128^{3}\) equal-mass particles at constant density \(n=0.5\,{\rm cm^{-3}}\) and temperature 10 K and \(\gamma=5/3\); \(6.78\times10^{46}\,\) J of energy is added to the central \(64\) particles in a top-hat distribution. This triggers a blastwave with initial Mach number \(\sim1000\).

Initial conditions: sedov_ics.hdf5

Parameterfile: “sedov.params”

In Config.sh, enable: 

BOX_PERIODIC  
SELFGRAVITY_OFF  
EOS_GAMMA=(5.0/3.0)

The file sedov_exact.txt gives the analytic solution at time \(t=30\) Myr, for the density, temperature, and radial velocity, as a function of radial coordinate from the blastwave center. Be careful, if you compare temperature, that you convert correctly (the analytic solution assumes a fully ionized plasma for the mean molecular weight, for example). Note that, because the problem is self-similar, we can compare any Sedov problem, at any time (so long as the shock is still ‘inside’ the box), and should obtain an identical result (up to the effective resolution of the explosion: obviously, the further the shock propagates through more particles/cells, the better-resolved).

The Noh (Spherical Implosion) Problem

Next consider the Noh 1987 test. This is a challenging test: many codes cannot run it without crashing, or run it but see catastrophic examples of the carbuncle instability. Liska & Wendroff 2003 noted only four of eight schemes they studied could actually run the problem. A large domain is initialized with \(\rho = 1\), \(\gamma=5/3\), vanishing pressure, and a radial inflow (directed towards the origin from all points) with \(|{\bf v}| = 1\) (\(v_{r} = -1\)). The analytic solution involves an infinitely strong shock with constant internal density moving outwards at speed \(=1/3\), with a density jump of \(4^{3}=64\) at the shock in 3D.

Initial conditions: noh_ics.hdf5

Parameterfile: “noh.params”

In Config.sh, enable: 

BOX_PERIODIC  
SELFGRAVITY_OFF  
EOS_GAMMA=(5.0/3.0)

This setup should not be compared to the analytic result much past time \(t=2\). The reason is that we do not use an inflow boundary condition, we just initialize a very large box. So at much later times, youve run out of inflowing material!

In the paper, we compare the density as a function of radius from the center of the sphere, at time \(t=2\) to the analytic solution. The analytic solution should be both spherically symmetric (i.e. not show excessive noise or the ‘carbuncle’ instability common in grid codes along the grid axes), and should have density jump at position \(r=t/3\), where \(t\) is the time. Inside the jump the density should be constant \(=64\), outside it should be \(=(1 + t/r)^2\).

Fluid Mixing Tests

Kelvin Helmholtz Instabilities

The KH instability is a common fluid instability in astrophysics. But SPH methods, in particular, have trouble capturing the growth of the instability, and some SPH methods lead to unphysical break-up of the instability in its non-linear stages even when it is accurately captured in the linear growth phase. On the other hand, AMR methods tend to over-diffuse the KH rolls, suppressing small-scale modes and structure in the gas. Because these tests are so common, we include here two different versions of the test problem.

First, we consider a two-dimensional setup from McNally et al. 2011. This is a KH initial condition with a non-zero thickness surface layer, and seeded mode, designed to behave identically in the linear regime in all well-behaved methods (as opposed to some setups, which depend on numerical error to seed the KH instability initially). The initial density changes from \(\rho_{1}=1\) to \(\rho_{2}=2\), and the initial \(x\)-velocity changes from \(-0.5\) to \(+0.5\), across a contact discontinuity in the \(y\) direction (the un-perturbed state is uniform in the \(x\)-direction); the exact functional form is given in McNally et al (it is complicated but essentially just smooths the transition over a \(y\)-direction width of \(\approx0.025\)). The pressure is a constant \(P=5/2\) with \(\gamma=5/3\) throughout a periodic domain of size \(0< x <1\), \(0< y <1\). The system is also seeded with an initial \(y\) velocity mode (sinusoidal in the \(x\)-direction with wavelength \(=1/2\)), with amplitude \(\delta v_{y}^{0} = 0.01\).

Initial conditions: kh_mcnally_2d_ics.hdf5 or kh_mcnally_2dgrid_ics.hdf5

Parameterfile: kh_mcnally_2d.params

In Config.sh, enable: 

    BOX_PERIODIC
    BOX_SPATIAL_DIMENSION=2
    PREVENT_PARTICLE_MERGE_SPLIT
    SELFGRAVITY_OFF
    KERNEL_FUNCTION=3
    EOS_GAMMA=(5.0/3.0)

For this test problem, the linear KH growth timescale is \(t_{\rm KH}=0.7\). The ‘correct’ solution cannot be included in a simple table, but you should compare to the solutions in Figs. 13–15 of the methods paper (note that the periodic box can be arbitrarily shifted in the vertical/horizontal directions, so check if your result doesn’t exactly ‘line up’ to the choice used for plotting before you decide its wrong!). Those plots are 2D gas density plots, scaled as described in the paper. By time \(t=1\), the initial mode should be growing into the familiar KH ‘rolls’; by time \(t=1.5\) those rolls should start rolling onto themselves, and by \(t=3-5\), their should be multiple internal rolls of each. Finally, by time \(t=5-10\), the whole box should be going non-linear: the rolls should begin to overlap and the whole central fluid should begin to ‘kink’.

Note that the default setup of this problem is inviscid, so the small-scale structure is always ‘seeded’ by grid noise, and there is no “converged” solution in the non-linear structure (the structure should appear at infinitely small scales, at infinite resolution). Following Lecoanet et al. 2016 (MNRAS, 455, 4274), to actually obtain define-ably converged solutions, need to make the system have a finite Reynolds number. To do this, add finite viscosity. So experiment with the flags: CONDUCTION and VISCOSITY, which are controlled by the flags ‘ShearViscosityCoeff’, ‘BulkViscosityCoeff’, ‘ConductionCoeff’ (some examples are in the parameterfile).

Because it is considered in many papers, we also compare a different KH IC. Specifically, we consider the 3D KH test from the Wengen multiphase test suite (described in detail in Agertz et al. 2007). Briefly, in a periodic box with size \(256,\,256,\,16\,{\rm kpc}\) in the \(x,\,y,\,z\) directions (centered on \(0,\,0,\,0\)), respectively, \(\approx10^{6}\) equal-mass particles are initialized in a cubic lattice, with density, temperature, and \(x\)-velocity \(=\rho_{1},\,T_{1},\,v_{1}\) for \(|y|<4\) and \(=\rho_{2}\,T_{2},\,v_{2}\) for \(|y|>4\), with \(\rho_{2}=0.5\,\rho_{1}\), \(T_{2}=2.0\,T_{1}\), \(v_{2}=-v_{1}=40\,{\rm km\,s^{-1}}\). The values for \(T_{1}\) are chosen so the sound speed \(c_{s,\,2}\approx 8\,|v_{2}|\); the system has constant initial pressure. To trigger instabilities, a sinusoidal velocity perturbation is applied to \(v_{y}\) near the boundary, with amplitude \(\delta v_{y} = 4\,{\rm km\,s^{-1}}\) and wavelength \(\lambda=128\,{\rm kpc}\).

This is a particularly interesting problem to compare fixed-grid and Voronoi-mesh methods. If you run GIZMO in fixed-grid mode, however, you need to make sure you use an IC file where the particles are actually on a grid! For this reason, the IC kh_mcnally_2dgrid_ics.hdf5 lays the particles on a regular Cartesian lattice (of variable mass), with initialized particle velocities, designed for exactly these experiments.

Initial conditions: kh_wengen_ics.hdf5

Parameterfile: kh_wengen.params (pytest: test/kh_wengen)

In Config.sh, enable: 

BOX_PERIODIC  
BOX_LONG_X=32  
BOX_LONG_Y=32  
BOX_LONG_Z=2  
SELFGRAVITY_OFF  
EOS_GAMMA=(5.0/3.0)

You should see similar qualitative behavior in this test as in the previous KH test, though here the linear growth time is longer (\(=3.5\)). Compare this to Fig. 16 in the code paper (note the same caveat about plotting). Since there is no ‘smoothing’ of the initial boundary, the phase transition in this test should remain sharp well into the non-linear evolution.

For both versions of the KH test, the number of domain segments per task is chosen automatically, but on a high-resolution IC like this one you can experiment with DOMAIN_SEGMENTS_SCALE to trade load-balancing against memory and communication. You should also experiment with the kernel function and hydro solver, and the DEVELOPER_MODE parameters that are set for example in the default parameterfile.

Rayleigh-Taylor Instabilities

We now consider the Rayleigh-Taylor (RT) instability, with initial conditions from Abel 2011. In a two-dimensional domain with \(0< x <1/2\) (periodic boundaries) and \(0< y <1\) (reflecting boundary with particles at initial \(y<0.1\) or \(y>0.9\) held fixed for the non-grid methods), we take \(\gamma=1.4\) and initialize a density profile \(\rho(y)=\rho_{1}+(\rho_{2}-\rho_{1})/(1+\exp{[-(y-0.5)/\Delta]})\) where \(\rho_{1}=1\) and \(\rho_{2}=2\) are the density “below” and “above” the contact discontinuity and \(\Delta=0.025\) is its width; initial entropies are assigned so the pressure gradient is in hydrostatic equilibrium with a uniform gravitational acceleration \(g=-1/2\) in the \(y\) direction (at the interface, \(P=\rho_{2}/\gamma=10/7\) so \(c_{s}=1\)). An initial \(y\)-velocity perturbation \(v_{y} = \delta v_{y}\,(1+\cos{(8\pi\,(x+1/4))})\,(1+\cos{(5\pi\,(y-1/2))})\) with \(\delta v_{y}=0.025\) is applied in the range \(0.3< y< 0.7\).

Initial conditions: rt_ics.hdf5

Parameterfile: “rt.params”

In Config.sh, enable: 

BOX_PERIODIC  
BOX_BND_PARTICLES  
BOX_LONG_X=1  
BOX_LONG_Y=2  
BOX_SPATIAL_DIMENSION=2  
EOS_GAMMA=1.4  
SELFGRAVITY_OFF  
GRAVITY_ANALYTIC

​
And make sure you go into the file gravity/analytic_gravity.h and un-comment the line:

//GravAccel_RayleighTaylorTest();     // vertical potential for RT tests

within the routine add_analytic_gravitational_forces. Otherwise the code won’t know what to use for the analytic gravitational forces.

Like the KH test, there is no trivial analytic solution for this test. However, you should compare the evolution to that in Fig. 18 of the methods paper (there we plot the density, from \(=0.8-2.8\), scaled linearly). By \(t=1\), the seeded mode should grow to a large, visible “rising/sinking” fluid perturbation. By \(t>2\), the perturbation should develop the “plume” shape which should be very pronounced by \(t=3\) (i.e. the “mushroom”-like shape should be evident). At \(t>3\), secondary KH instabilities should become obvious along the edge of the rising/sinking shear surface. And at \(t\sim 4.5\) the sinking plumes should “hit” the bottom of the “live” domain.

You will find that SPH also has difficulty with this test as well as the KH test. It requires, even in the modern PSPH methods, going to larger neighbor number before reasonable results are obtained. Also, both in the KH and RT tests, PSPH sometimes appears to perform better without artificial conductivity terms, which introduce excessive diffusion and unphysically distort the structure of small-wavelength perturbations. However, such terms are necessary to reduce noise in strong shock problems. If you compare grid codes, boost the velocity by a modest amount – you’ll find at this resolution that the RT evolution is severely diffused and slowed-down. Comparing the MFM and MFV methods, you’ll find that contact discontinuities are sharper in the MFV method, so for a problem like this it has effectively “higher” resolution.

The Blob Test

Next we consider the “blob” test, which is designed to synthesize the fluid mixing instabilities above (as well as ram-pressure stripping) in a more “realistic” example of astrophysical interest representative of a multi-phase medium. The initial conditions come from the Wengen test suite and are described in Agertz et al. 2007. We initialize a spherical cloud of uniform density in pressure equilibrium with an ambient medium, in a wind-tunnel with period boundaries. The imposed wind has Mach number \(\mathcal{M}=2.7\) (relative to the “ambient” gas) with the cloud having a density \(=10\) times larger than the ambient medium. The domain is a periodic rectangle with dimensions \(x,\,y,\,z=2000,\,2000,\,6000\,\) kpc (the absolute units are not important), with the cloud centered on \(0,\,0,\,-2000\,\) kpc; \(9.6\times10^{6}\) particles/cells are initialized in a lattice (with equal-masses in the particle-based methods).

Initial conditions: blob_ics.hdf5 (Note: this IC file is much larger than most others)

Parameterfile: “blob.params” (pytest: test/blob)

In Config.sh, enable: 

BOX_PERIODIC  
BOX_LONG_X=1  
BOX_LONG_Y=1  
BOX_LONG_Z=3  
SELFGRAVITY_OFF  
EOS_GAMMA=(5.0/3.0)

The number of domain segments per task is chosen automatically, but on a high-resolution IC like this one you can experiment with DOMAIN_SEGMENTS_SCALE to trade load-balancing against memory and communication.

Qualitatively, if you make images of the gas density through the \(y=0\) plane, you should see a bow shock form around the cloud, while a slower shock propagates into the cloud. Eventually the cold, dense material is gradually stripped and shredded into small bits which are destroyed by a combination of RT and KH instabilities. Compare to e.g. Fig. 20 in the code paper.

A more quantitative comparison is formulated as follows. At each time, measure the total mass in gas with \(\rho>0.64\,\rho_{c}\) and \(T<0.9\,T_{a}\) (where \(\rho_{c}\) and \(T_{a}\) are the initial cloud density and ambient temperature). Then normalize this by dividing by the mass you get in the initial condition at \(t=0\). This is one way to define a “surviving” cloud mass. This is plotted as a function of time \(t\), or \(t/\tau_{\rm KH}\) where \(\tau_{\rm KH}=2\) is the KH instability growth time across the cloud. Compare this to the results in Fig. 21 of the code paper.

Gravity and Cosmological Tests

The next set of test problems all involve self-gravity, coupled to hydrodynamics, and in some cases, cosmological integration.

The Evrard (Spherical Collapse) Test

This problem comes from Evrard 1988, and is commonly used to test SPH codes, and more recently, grid codes as well. In an infinite, open domain, we initialize a three-dimensional sphere of gas with adiabatic index \(\gamma=5/3\), mass \(M=1\), radius \(R=1\), and initial density profile \(\rho(r) = M/(2\pi\,R^{2}\,r) = 1/(2\pi\,r)\) for \(r < R\) and \(\rho=0\) outside the sphere. The gas is initially at rest and has thermal energy per unit mass \(u=0.05\) (much less than the gravitational binding energy).

Initial conditions: evrard_ics.hdf5

Parameterfile: “evrard.params” (pytest: test/evrard)

In Config.sh, enable: 

EOS_GAMMA=(5.0/3.0)

You should experiment with enabling/disabling ADAPTIVE_GRAVSOFT_FORGAS, and varying the gravitational softenings (remember, with ADAPTIVE_GRAVSOFT_FORGAS set, the softenings in the parameterfile for gas are the minimum values; otherwise they are constant through the run). See how this changes the results. You may also want to experiment with DOMAIN_SEGMENTS_SCALE, which trades load-balancing against memory and communication.

When the simulation begins, the gas free-falls towards \(r=0\) under self-gravity, until a strong shock occurs and the inner regions “bounce” back, sending the shock outwards through the infalling outer regions of the sphere. Eventually, the shock propagates across the whole sphere and the system settles into a hydrostatic virial equilibrium. The test is useful because it is typical of gravitational collapse of structures, and because it involves the conversion of gravitational energy to kinetic energy then to thermal energy; so it is quite sensitive to the total energy conservation of the code (particularly challenging for coupled gravity-hydro methods with adaptive timestepping, as we use here).

The file evrard_exact.txt has a solution from a well-vetted, high-order (and much-higher resolution) simulation at time \(t=0.8\); specifically we compare the density, entropy (defined by the entropic function \(s=P/\rho^{gamma}\) ), and radial velocity of the flow as a function of radius from the center of the sphere. You should find that, because the ICs here are not perfectly smooth, all methods exhibit some noise. But the noise and smoothing of the entropy jump in particular are much more noticeable in SPH, especially when an artificial conductivity is used. Fixed-grid codes will also lose resolution as the sphere collapses.

The Zeldovich Pancake

A standard test for cosmological integration is the “Zeldovich pancake”: the evolution of a single Fourier mode density perturbation in an Einstein-de Sitter space. This is a useful test both for scientific reasons (it represents a ‘single mode’ of large-scale structure formation in cosmology) and for testing a code’s ability to deal with cosmological integrations, small-amplitude perturbations, extremely high Mach-number flows and shocks, and highly anisotropic cell/particle arrangements. Following Zeldovich 1970: we initialize a small sinusoidal (wavelength \(\lambda=64\,h^{-1}\) Mpc) perturbation in a periodic, cosmological box (see the code paper for details of the setup), with a cold initial temperature (chosen so the pressure forces are negligible). The perturbation is designed so that it collapses into a caustic (density divergence) at finite redshift \(z_{c}=1\). The box is 3D, but the the perturbation is just initialized in the \(x\)-direction, which defines the direction in which collapse occurs. We initialize the particles in a glass, the ‘standard practice’ for cosmological runs.

Initial conditions: zeldovich_ics.hdf5

Parameterfile: “zeldovich.params” (pytest: test/zeldovich)

In Config.sh, enable: 

BOX_PERIODIC  
PMGRID=256 (recommended, see below)  
EOS_GAMMA=(5.0/3.0)

You should experiment with enabling/disabling ADAPTIVE_GRAVSOFT_FORGAS, and varying the gravitational softenings (remember, with ADAPTIVE_GRAVSOFT_FORGAS set, the softenings in the parameterfile for gas are the minimum values; otherwise they are constant through the run). See how this changes the results. You may also want to experiment with DOMAIN_SEGMENTS_SCALE, which trades load-balancing against memory and communication. You can turn off PMGRID and run in pure tree mode (slower, but should work as well or slightly more accurately), or vary the PMGRID level between e.g. 64–256.

In early phases, \(z > z_{c}\) (when pressure forces are negligible), the system simply traces the linear solution given above: this is captured well by all methods. The interesting dynamics occur after the caustic formation at \(z_{c}\): the caustic collapses and forms a strong shock (factor \(\sim 10^{10}\) temperature jump!), which propagates outwards, with a central temperature cavity that has (formally) divergent density at \(x=0\) as the external pressure/temperature vanishes (\(T_{i}\rightarrow0\)). The un-shocked flow follows the extension of the linear solution. The file zeldovich_exact.txt contains an approximately exact (1D, high-resolution, PPM calculation) solution at \(z=0\), for the gas density, temperature, and x-velocity. You should compare to your results; you’ll find more noise (expected, because of the glass ICs; if we used a grid IC like this calculation does the initial density field would be smooth, but we didn’t, because it’s not stable for real cosmological problems). But you should find some of the same systematic differences between methods (SPH smoothing more, for example).

We note that many codes only test this problem in 1D, but it is actually much more challenging (and representative of the resolution that can be achieved in real simulations) to test it in 3D, as we do here.

The Santa Barbara Cluster

We next consider the “Santa Barbara Cluster” from the comparison project in Frenk 1999. This is a standard reference test problem for which many codes have been compared. It is a zoom-in simulation in which a low-resolution cosmological background contains a higher-resolution Lagrangian region which will collapse to form an object of interest (and the region around it) by \(z=0\); here chosen so the object represents a rich galaxy cluster in an Einstein-de Sitter Universe. The details of the cluster ICs are described there; briefly, a periodic box of side-length \(64\,h^{-1}\,{\rm Mpc}\) is initialized at redshift \(z=49\) (\(a=1/(1+z)=0.02\)), in a flat Universe with dark matter density \(\Omega_{\rm DM}=0.9\), baryonic \(\Omega_{\rm b}=0.1\), Hubble constant \(H_{0}=100\,h\,{\rm km\,s^{-1}\,Mpc^{-1}}\) with \(h=0.5\), and negligible initial gas temperature \(T=100\,\) K. The gas is non-radiative (ideal) with \(\gamma=5/3\).

Initial conditions: sbcluster_ics.hdf5

Parameterfile: “sbcluster.params”

In Config.sh, enable: 

BOX_PERIODIC  
PMGRID=64 (recommended, see below)  
EOS_GAMMA=(5.0/3.0)

You should experiment with enabling/disabling ADAPTIVE_GRAVSOFT_FORGAS and ADAPTIVE_GRAVSOFT_FORALL or ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION, and varying the gravitational softenings (remember, with ADAPTIVE_GRAVSOFT_FORGAS set, the softenings in the parameterfile for gas are the minimum values; otherwise they are constant through the run). See how this changes the results. You may also want to experiment with DOMAIN_SEGMENTS_SCALE, which trades load-balancing against memory and communication. You can turn off PMGRID and run in pure tree mode (slower, but should work as well or slightly more accurately), or vary the PMGRID level between e.g. 64–256.

Parts of the ‘exact’ solution for this problem are still the subject of considerable controversy in the community, so we don’t want to provide a single high resolution calculation and claim that it’s “correct.” However, you should be able to make some basic comparisons at \(z=0\). The dark matter profile can be compared to a Navarro, Frenk, and White profile, with virial radius \(R_{\rm vir} = 2.75\,h^{-1}\) Mpc, and concentration parameter \(c=7.5\) (the normalization of the profile then follows from the adopted cosmology). Note that this isn’t an exact analytic solution, it’s just a fitting function, but it does quite well for high-res calculations – if you’re seriously deviating from this, it indicates a real problem in the gravity or hydro solution. The gas follows a similar solution, suppressed by the ratio of baryons to dark matter \(=0.1/0.9\), down to smaller radii \(\sim 0.2\,h^{-1}\) Mpc, where it flattens relative to the NFW profile because of pressure support.

The more controversial aspect of the profile is the central entropy, where people plot \(\ln{s}\) (where \(s\) is the mass-weighted entropy in concentric spherical shells, with \(s=T/\rho^{2/3}\) with \(T\) in units of Kelvin, assuming fully ionized plasma so mean molecular weight \(=0.59\), and \(\rho\) in units of solar masses per cubic Mpc) as a function of radius. Around the scale where the density profile levels out, these profiles begin to diverge with different numerical methods – fixed-grid codes tend to give a minimum central “core” with \(\ln{s}=-2\) being very flat, while TSPH leads to a more power-law like central profile where \(\ln{s}\) decreases to values of –4 or –5 at \(0.01\,h^{-1}\) Mpc. What you should see is that, if you play around with the numerical method used to solve the hydrodynamics, the gravitational force softening, the choice of Riemann solver, the slope-limiter used in the Riemann problem, and the choice to use (or not use) adaptive gravitational softenings, is that you can reproduce this whole range of behavior. In general, if you make your solver more diffusive, you get a larger entropy core; but even more striking is that if you make the small-scale particle noise in the gravitational field larger (by making the gravitational softening too small or using too-aggressive adaptive softenings), you get larger cores. The large cores seen in AMR codes appear to be artificially generated by small-scale particle noise in the gravitational field seeding weak motions of the fluid relative to the grid, while leads to small shocks and mixing entropy. SPH totally suppresses this, at the cost of suppressing some real mixing. Qualitatively, moving Voronoi-mesh codes (Springel et al. 2010), our MFM and MFV calculations, and the ‘improved/modern’ SPH calculations (see e.g. those here and Saitoh and Makino, 2010, 2012), all agree the ‘correct’ answer lies in-between the old SPH and AMR results, probably closer to the old SPH results but still with significant uncertainties.

Galactic Disks with and without ISM Physics

This last test is not really a ‘test’ (in the sense that there is no ‘right’ answer) – it is more a useful example with which to compare different physics, and different methods for solving the hydrodynamics and gravity. The initial condition is an isolated disk galaxy, designed to be run in an open domain. The disk is a very low-resolution analogue of a Milky-Way like galaxy. It has a Hernquist-profile bulge (particle type =3), a stellar disk with initial Toomre Q=1 (particle type=2), an extended Hernquist-profile, isotropic dispersion supported dark matter halo (particle-type=1), a thin gas disk (particle-type=0), and a central super-massive black holes (particle-type=5). If star formation is enabled, the new stars formed from the gas will occupy particle-type=4. The gas and stellar disk are rotating in approximately circular orbits (corrected for the initial gas pressure and stellar velocity dispersion). The disk – even evolved as an ideal gas – will develop interesting structure because of self-gravity, and if cooling, star formation, stellar feedback, and/or black hole growth and feedback are enabled, a variety of interesting dynamics can be studied. This represents just one of the kinds of complicated, multi-physics problems for which GIZMO is designed.

The point is, the code is a playground, explore and have fun!

Initial conditions: isodisk_ics.hdf5

Parameterfile: “isodisk.params”

For the “ideal gas” (no star formation or cooling) version of the test, in Config.sh, enable: 

EOS_GAMMA=(5.0/3.0)

Try doing the same, now with high-temperature cooling above \(10^{4}\) K assuming an optically thin, primordial H-He mixture: 

EOS_GAMMA=(5.0/3.0)  
COOLING

If you want to compare the Springel and Hernquist ‘effective equation of state’ (approximate sub-grid star formation and stellar feedback) models, then enable in Config.sh:

EOS_GAMMA=(5.0/3.0)  
COOLING  
GALSF  
METALS  
GALSF_EFFECTIVE_EQS

You should experiment with enabling/disabling ADAPTIVE_GRAVSOFT_FORGAS and ADAPTIVE_GRAVSOFT_FORALL or ADAPTIVE_GRAVSOFT_FROM_TIDAL_CRITERION, and varying the gravitational softenings (remember, with ADAPTIVE_GRAVSOFT_FORGAS set, the softenings in the parameterfile for gas are the minimum values; otherwise they are constant through the run). See how this changes the results. You may also want to experiment with DOMAIN_SEGMENTS_SCALE, which trades load-balancing against memory and communication.

Collisionless and Few-Body Gravity Tests

Pure-gravity collisionless and few-body problems, complementing the hydro-plus-gravity tests of the tests above. Several tolerances in these tests are calibrated from recorded reference runs, and the reasoning is documented in the test files and READMEs. Read those before treating a marginal failure as a regression.

Hernquist Sphere (test/hernquist): A \(2^{15}\)-particle equilibrium Hernquist sphere (\(G=M=a=1\)), run to \(t=118\) (about 10 crossing times) under eight gravity-solver variants: baseline, ADAPTIVE_GRAVSOFT_FORALL, BOX_PERIODIC+PMGRID (TreePM), BOX_PERIODIC (Ewald), TIDAL_TIMESTEP_CRITERION, tidal timestepping with tidal-radius adaptive softening, and RANDOMIZE_GRAVTREE with and without TreePM. Each variant must keep its half-mass radius within 10% and its total energy within a per-variant ceiling (3x a recorded reference). The spurious centre-of-mass drift must stay bounded, and the RANDOMIZE_GRAVTREE variants are compared against their unrandomized partners. The two unrandomized periodic variants are marked as a known expected failure on the half-mass radius. A second file, test_hernquist_convergence.py, scans ErrTolIntAccuracy for the acceleration and tidal timestep criteria. It requires the tidal criterion’s accumulated energy error to be within a factor of 2 of the acceleration criterion’s at the default tolerance, and to fall as the tolerance is tightened (see README_convergence.md). The same directory holds optional dark-matter physics examples on this halo, outside the pytest sweep: self-interacting dark matter (Config_sidm.sh, hernquist_sidm.params), fuzzy dark matter (Config_dmfuzzy.sh, hernquist_dmfuzzy.params) and the continuum Vlasov integrator (Config_cbe.sh, hernquist_cbe.params). Their params choose units of kpc, \(10^{10}\,M_{\odot}\) and 207.4 km/s, for which \(G=1\) exactly, so the same IC generator gives a \(10^{10}\,M_{\odot}\), \(a=1\) kpc halo with cross-sections and boson masses in physical units; validate_snapshot_vs_ic.py compares any snapshot with the IC (see README_dm_physics.md).

Plummer Sphere and Star Clusters (test/plummer): test_plummer.py runs a \(2^{15}\)-particle equilibrium Plummer sphere through the same eight gravity variants as test/hernquist. It requires the 10%, 50% and 90% Lagrange radii to be preserved, energy to be conserved to 4% of the initial kinetic energy, and the secular energy drift rate to stay within 3x a per-variant reference. test_plummer_binaries.py replaces each of 256 Plummer particles with an equal-mass circular binary of sink particles (Hermite integration, few-body timestepping), and compares tree gravity with SINGLE_STAR_DIRECT_GRAVITY (exact star-star summation). It requires the half-mass radius to hold to 15% and energy to be conserved to 1%. test_plummer_binaries_realistic.py instead draws a heterogeneous population (Kroupa IMF, mass-dependent binary fraction, log-normal periods, thermal eccentricities, truncated at 100 AU pericentre) and bounds the energy error and centre-of-mass drift over 10 crossing times (see README_plummer_binaries_realistic.md). All binary diagnostics use the synchronized IO_HERMITE_SYNC positions and velocities, with the potential recomputed by direct summation.

Few-Body Integration (test/fewbody): Tests of the Hermite integrator for sink particles (SINGLE_STAR_STARFORGE_DEFAULTS + IO_HERMITE_SYNC) on exactly known few-body systems. test_binary.py evolves an \(e=0.9\), \(q=0.1\) Kepler binary. It requires the relative energy error to stay below \(1.5\times10^{-3}\) and the spurious centre-of-mass velocity below \(5\times10^{-4}\) of the orbital velocity (the ICs have zero total momentum exactly). It also scans ErrTolIntAccuracy and requires the energy error to converge at better than first order. A DISABLE_HERMITE_INTEGRATION (KDK leapfrog) comparison variant runs only on request (marked slow). test_triple.py evolves an inclined, eccentric, unequal-mass hierarchical triple whose inner binary and tertiary sit about 6 timebins apart, and bounds the per-outer-orbit energy error at \(1.7\times10^{-4}\). That is a gross-breakage ceiling only; see README_triple.md for what it does and does not guard.

Gravity Tree Accuracy

Tree Force Accuracy (test/gravtree): The classic tree-force error test. Each variant is built once and run for a single step twice from the same IC: with the parameterfile’s own (production) opening criterion, and with the walk tightened towards direct summation (ErrTolTheta=0.15, ErrTolForceAcc\(=10^{-6}\)). The two snapshot_000 files, which hold the start-up force evaluation, are compared particle by particle (matched by ID), so the difference is the tree error alone. Five variants share one Config.sh plus per-variant flags: the Evrard IC with fixed softening; the same with the potential accumulated on the walk (EVALPOTENTIAL); a periodic box with the TreePM split (BOX_PERIODIC, GRAVITY_NOT_PERIODIC, PMGRID=64, on the gmc_cooling IC); and the isolated-disk IC with sink distances (SINK_CALC_DISTANCES) or long-range stellar radiation (GALSF_FB_FIRE_RT_LONGRANGE) gathered on the walk. For the gravitational acceleration (OUTPUT_ACCELERATION with OUTPUT_HYDROACCELERATION, so the hydro term is excluded) and the potential, the tight walk is a converged reference, and the test bounds the median and 99th percentile of the per-particle relative error (measured: about 0.1–0.3% median and 0.2–0.4% at the 99th percentile); a defect in the walk – a missed node, a wrong softening or periodic image – appears as a large error on some subset of particles even when the median stays small. The sink distance and the radiation energy are gathered opportunistically on a walk whose openings are chosen for force accuracy alone, so for them the tight run is only a tighter-walk reference: Sink_Distance is compared only where both walks report one (beyond the short-range region, whether a cell meets the sink at all depends on which nodes are opened) and must agree exactly there, while PhotonEnergy, a deliberately crude long-range estimate with no converged solution, is gated only on its median and energy-weighted error (a heavy per-cell tail is expected).

Group and Halo Finding

FOF/SUBFIND Halo Finding (test/fof_subfind): Runs the friends-of-friends group finder and SUBFIND (FOF, SUBFIND, linking type 1) in post-processing mode (RestartFlag=3) on a locally generated periodic particle set: one FOF halo made of two bridged density peaks, plus a separate compact halo. The test checks that the neighbour-list FOF and SUBFIND stages report completion, and that the group catalogue holds at least two groups (the largest with at least 200 members) and at least two subhalos (the largest with at least 40). It is a functional check of the halo-finding pipeline, not a comparison against a reference catalogue.

Continuum Vlasov (Collisionless Boltzmann) Tests

These problems exercise the continuum Vlasov integrator (CBE_INTEGRATOR; see the Config section; Hopkins 2026, in preparation), in which each collisionless particle carries a mixture of velocity-space basis functions (CBE_INTEGRATOR_SECONDMOMENT, CBE_INTEGRATOR_WITHGRADIENTS). The ICs (including the per-basis VlasovMoments) are generated locally by each test’s make_ic.py. Unless noted, each test asserts the same conservation and diagnostic gates:

Each test writes per-stream density profiles compared with exact collisionless advection, but these plots are diagnostic only and not gated. The 1D tests run on 2 MPI ranks and on 1 rank with 2 OpenMP threads.

CBE Counter-Streaming Beams (test/cbe_two_stream): In a 1D periodic box, each particle carries two cold velocity bases at \(\pm v\) with zero bulk velocity (CBE_INTEGRATOR=2), so the two streams advect through each other while the total density stays uniform. Because the problem is symmetric, momentum and the second moment must be conserved to round-off (\(10^{-10}\) and \(10^{-9}\) respectively) in addition to the common gates.

CBE Density Wave (test/cbe_density_wave): As test/cbe_two_stream, but with a cosine density modulation of amplitude 0.2 (\(\rho = 1 + \epsilon\cos 2\pi x/L\)), so each stream carries the perturbation and the analytic solution is the IC density translated at \(\pm v\). It applies the common gates, with momentum conserved to \(10^{-10}\) and the second moment to \(10^{-4}\).

CBE Free-Slot Injection, 1D (test/cbe_free_slot_1d): Four cold streams (\(v = +1, 0, -1, -2\); CBE_INTEGRATOR=4) plus a localized Gaussian perturbation at \(v=+2\) that neighbouring particles have no basis for. Advecting it requires the free-slot pairing fallback, which opens a new basis to receive the incoming flux. The common conservation and face-residual gates are asserted. The masses in each velocity band are printed, and a plot of the per-band density shows whether the \(+2\) perturbation is carried correctly.

CBE Free-Slot Injection, 3D (test/cbe_free_slot_3d): The same streams and perturbation as test/cbe_free_slot_1d on a 3D slab (BOX_LONG_X=16, velocities along \(x\)). It checks that free-slot routing and conservation survive the full 3D moment set and domain decomposition, running on 2 and 4 MPI ranks with looser momentum and second-moment tolerances (\(3\times10^{-3}\)).

CBE Harmonic Breathing Mode (test/cbe_harmonic_1d): A hot 1D Gaussian distribution in a fixed external harmonic potential (GRAVITY_ANALYTIC_HARMONIC), with the exact one-sided Gaussian-basis Riemann solver (CBE_INTEGRATOR_RP_GAUSSIAN). A Gaussian \(f\propto\exp[-(v^{2}+\Omega^{2}x^{2})/2\sigma^{2}]\) is an exact stationary solution, and the IC’s spatial width is set to 0.7 of the equilibrium value. The analytic response is therefore an undamped oscillation of the width at \(2\Omega\). The run must reach its end time, conserve mass to \(10^{-10}\), and keep the full compression and expansion range of the oscillation without damping or runaway. The per-basis velocities must also stay bounded, which catches basis collapse.

The same integrator can also be run on a self-gravitating, three-dimensional equilibrium: see the continuum Vlasov example on the Hernquist halo in Collisionless and Few-Body Gravity Tests.

Magneto-Hydrodynamics Tests

With the release of the MHD methods paper and public version of the MHD code, I have added a suite of MHD test problems. The initial conditions, example parameter files, and list of compile-time (Config.sh) flags which must be set to run each problem are publicly available at the same location as the rest of the files above.

The details of these problems, descriptions of their setups, and the results (to which your runs can and should be compared), are all given in the MHD GIZMO methods paper: Hopkins & Raives 2015.

The test problems included are:

  1. Linear magnetosonic wave (mhd_wave_ics.hdf5, mhd_wave.params) – test/mhd_wave
  2. Brio-Wu shocktube (briowu_ics.hdf5, briowu.params) – test/briowu
  3. Toth shocktube (toth_ics.hdf5, toth.params) – test/toth
  4. Field loop advection (field_loop_ics.hdf5, field_loop.params) – test/field_loop
  5. Hawley-Stone current sheet (currentsheet_A0pt1_b0pt1_ics.hdf5, currentsheet_A0pt1_b1em12_ics.hdf5, currentsheet_A1e4_b0pt1_ics.hdf5, currentsheet.params) – test/currentsheet
  6. Orszag-Tang vortex (orszag_tang_ics.hdf5, orszag_tang.params) – test/orszag_tang
  7. Balsara-Spicer MHD rotor (rotor_ics.hdf5, rotor.params) – test/rotor
  8. Blastwave in a magnetized medium (mhd_blast_ics.hdf5, mhd_blast.params) – test/mhd_blast
  9. Rayleigh-Taylor instability with MHD (rt_ics.hdf5, rt.params) – test/rt
  10. Kelvin-Helmholtz instability with MHD (kh_mcnally_2d_ics.hdf5, kh_mcnally_2d.params) – test/kh_mcnally_2d
  11. Magneto-Rotational instability in a shearing sheet (mri_ics.hdf5, mri.params) – test/mri
  12. Magnetized core-collapse and jet-launching (core_ics.hdf5, core.params) – test/core
  13. Zeldovich pancake with MHD (zeldovich_mhd_ics.hdf5, zeldovich_mhd.params) – test/zeldovich_mhd
  14. Santa Barbara cluster with MHD (sbcluster_ics.hdf5, sbcluster.params) – test/sbcluster
  15. Isolated galaxy disk with MHD (isodisk_ics.hdf5, isodisk.params) – test/isodisk

In the list above, parenthesis contain the name of the ICs file and name of the example parameterfile. Just like all of the examples above, this should help you set up and get a problem running, but you should not take any parameters as recommendations or as exactly matching published solutions (rather, you should explore different choices yourself).

For convenience, rather than list the recommended compile-time options (for Config.sh) here, each of the parameter files listed above actually now includes these in the header of the actual params file. As always, you can freely vary the hydro method, but all the required flags which must be set at compile time should be listed. You still need to manually set them in Config.sh and compile for each problem, but the params file should now provide all the information you need (including any special instructions for some problems that require lines in the source code be modified).

Note that the Rayleigh-Taylor, Kelvin-Helmholtz, Zeldovich pancake, Santa Barbara cluster, and isolated galaxy disk setups are essentially identical to their non-MHD versions, and in most of those cases it is the same IC file needed. You simply compile the code with MAGNETIC turned on (and potentially some other parameters to set the initial magnetic fields, which should be given in the example files above). The other tests are self-contained.

For many of the problems (any with MHD_B_SET_IN_PARAMS as a recommended compile-time option), you can freely re-set the initial magnetic field by changing the values of BiniX, BiniY, BiniZ in the parameter file. This allows you to easily explore the effects of varying field strength.

The current sheet test is the only one with multiple initial conditions: as discussed in the paper, the challenge is exploring different initial values of the parameters \(A\) and \(\beta\). The IC “currentsheet_A0pt1_b0pt1_ics.hdf5” assumes the easy values \(A=0.1\), \(\beta=0.1\). The IC “currentsheet_A0pt1_b1em12_ics.hdf5” assumes \(A=0.1\), \(\beta=10^{-12}\), i.e. incredibly strong magnetic fields. The IC “currentsheet_A1e4_b0pt1_ics.hdf5” assumes \(A=10^{4}\), \(\beta=0.1\), i.e. incredibly large Mach numbers.

As above, please explore these freely.

Non-Ideal MHD and Multi-Fluid Tests

These problems exercise the non-ideal terms (MHD_NON_IDEAL: Ohmic resistivity, the Hall effect and ambipolar diffusion), the battery source terms, and the ion-neutral coupling of the multi-fluid framework. Each compares against an analytic or reference solution.

C-Type Shock (test/c_shock): A 1D C-type (continuous) shock in a weakly-ionized medium, where ambipolar diffusion broadens the jump into the smooth profile characteristic of ion-neutral drift (rather than a sharp J-shock). Uses MAGNETIC, MHD_NON_IDEAL, EOS_ENFORCE_ADIABAT, BOX_SPATIAL_DIMENSION=1, with problem-specific coefficients (\(\eta_{\rm AD} = 10^{5}\,v_{A}^{2}\), \(\eta_{\rm Ohm} = 10^{-5}\,\eta_{\rm AD}\), no Hall term). The final density, internal energy, velocity and magnetic field are compared against a reference numerical solution (c_shock_exact.hdf5) to a loose relative tolerance of 10%.

Hall MHD Waves (test/hall_wave): Propagation of whistler/ion-cyclotron waves in 1D with fixed non-ideal coefficients (\(\eta_{\rm Hall}=0.01\), \(\eta_{\rm Ohm}=2\times10^{-4}\), no ambipolar term), following Berlok & Pfrommer (2023, arXiv:2309.15907). Uses MAGNETIC, MHD_NON_IDEAL, BOX_SPATIAL_DIMENSION=1. The final velocities and magnetic field are compared against a reference solution (hall_wave_exact.hdf5) to 5%, and a plot of the field and velocity components of one tracked cell versus time is written for inspection.

Biermann Battery Growth (test/biermann_growth): A 2D periodic box with \(\mathbf{B}=0\) and orthogonal sinusoidal gradients in density and temperature, from which the electron Biermann battery must generate a \(B_{z}\) field. Uses MAGNETIC, MHD_BATTERY_MECHANISMS=1, MHD_B_SET_IN_PARAMS, BOX_SPATIAL_DIMENSION=2; the IC is generated locally by make_biermann_growth_ics.py. The test compares the field near the source maximum against the analytic rate \(\mathrm{d}B_{z}/\mathrm{d}t = -(c\,k_{B}/e)\,a\,b\,T_{0}\,k^{2}\), requiring the correct sign and a magnitude within a factor of 5. It is a sign and order-of-magnitude check on the battery source term, not a precision convergence test.

Ion-Neutral Ambipolar Drag Relaxation (test/ionneutral_drag): Two interleaved Type=0 fluids in a 1D periodic box — 32 ions tagged FluidType=FLUID_ION (with \(v_x = +10^{-4}\) and a small \(B_x\)) interleaved with 32 neutrals tagged FluidType=FLUID_DEFAULT (\(v=0\), \(B=0\)) — relax exponentially via the Draine ambipolar drag with \(\mathrm{d}(\Delta v)/\mathrm{d}t = -\gamma_{\rm AD}(\rho_i+\rho_n)\,\Delta v\). Code units are chosen so \(\gamma_{\rm AD,code}=1\) and \(\rho_{\rm total}=1\), yielding analytic \(t_{\rm damp}=1\) in code time. The pytest gate fits the measured decay rate to \(\exp(-t)\) across 5 e-folds and asserts \(\max|B_{\rm neutral}|<10^{-12}\) throughout (the cross-fluid MHD terms are skipped for neutrals, which therefore stay at \(B=0\) under induction). The IC is generated locally by make_ionneutral_drag_ics.py (no website download). A recorded run measures a decay rate of 0.9997 against the analytic 1 over the full 5 e-folds. Uses HYDRO_MULTIFLUID, HYDRO_MULTIFLUID_IONNEUTRAL, MAGNETIC, HYDRO_MESHLESS_FINITE_MASS, BOX_SPATIAL_DIMENSION=1.

Solid-Body, Elasto-Dynamics, and Equation-of-State Tests

The elastic and solid-body portions of the code (elastic stresses, the Tillotson and tabulated M-ANEOS equations of state, porosity, damage and fracture) have been vetted on a number of simple wave tests and on the problems below, which range from colliding elastic rings and impact cratering to self-gravitating planetary bodies and tabulated-EOS shock tubes. Since there is not a formal methods paper for several of these modules, users are encouraged to develop additional test problems and contribute them, as well as to explore and expand this portion of the code functionality.

Bouncing Rubber Cylinders

This is the colliding/bouncing rubber ring/cylinder test studied in a large number of papers (e.g. Swegle ‘SPH in Tension’, memo, Sandia National Laboratories, 1992; Sulsky et al., 1995, Comp. Phys. Comm; Monaghan 2000, J. Comp. Phys; Gray et al., 2001, Comp. Methods. Appl. Mech. Eng.). Two rings (between annuli \(r=3-4\), in code units) in 2D (representing infinite cylinders) with density \(\rho_{0}\) and equation-of-state \(P=c_{s}^{2}(\rho-\rho_{0})\), approach at velocity \(\pm 0.059\,c_{s}\), collide, bounce, and move away, oscillating (deforming) internally as they do. In code units, \(c_{s}=\rho_{0}=1\), and the shear modulus of the material is set to \(\mu=0.22\). The desired equation of state is achieved by setting the appropriate parameters for the more general custom Tillotson equation-of-state.

This is a powerful and useful test because while simple, it is actually quite demanding in several respects. Most notably, many algorithms suffer from the “tensile instability” and the rings artificially “crack” or fragment on their collision. If the algorithm is too dissipative, the rings will “stick” rather than bounce. Very small asymmetries in the initial conditions are amplified strongly in the “bounce,” so the problem tests the symmetry of the code algorithms. If the elastic forces are not correctly computed, the rings will have no restoring forces and will not oscillate after the bounce. If the equation of state and hydro solver cannot properly handle negative pressures, the rings will diffuse outwards before and after collision.

Initial conditions: “ring_collision_ics.hdf5”

Parameterfile: “ring_collision.params”

The pytest version (test/ring_collision) runs this setup and requires the run to reach its end time with mass conserved to 0.1%, and the rings to have bounced apart rather than merged (their final \(x\)-extent at least half the initial one).

In Config.sh, enable: 

HYDRO_MESHLESS_FINITE_MASS  
BOX_SPATIAL_DIMENSION=2  
SELFGRAVITY_OFF  
EOS_TILLOTSON  
EOS_ELASTIC  
KERNEL_FUNCTION=6

The choice of KERNEL_FUNCTION=6 is optional here. Experiment with different kernels, but be sure to set the parameter DesNumNgb accordingly; for e.g. KERNEL_FUNCTION=3, use 20, for KERNEL_FUNCTION=4 or KERNEL_FUNCTION=6 use 32, for KERNEL_FUNCTION=5 (or 7), use 40. SPH, for example, performs noticably better using the Wendland kernels on this test (while for MFM it makes a much smaller difference). The choice of hydro solver is also arbitrary – however, recall that because of the material assumptions, the elasto-dynamics modules in the code are only designed to work with fixed-mass methods (MFM or SPH variants) at present. But comparing MFM and SPH shows interesting differences: SPH may have advantages in some limits of very stiff EOS’s. Within SPH, you should primarily use the Density-SPH formulation (HYDRO_DENSITY_SPH): the Pressure-SPH formulation will formally work, but is really designed to capture certain types of fluid behaviors that don’t really apply in the elastic limit.

Note that this problem can be sensitive to the ICs: a setup without good ICs will have cracks along the surface or irregularities that can either lead to the rings ‘sticking’ or fracturing. The code attempts to deal with the sharp boundaries in the ICs with multiple methods, as well. By default, we use the kernel volume correction for sharp boundaries from Reinhardt & Stadel 2017 (arXiv:1701.08296), i.e. HYDRO_KERNEL_SURFACE_VOLCORR is turned on by default (you can turn if off if desired), which allows for a sharped (single-cell) density gradient. We also use a modified version for elasto-dynamics by default of directly integrating the kernel volume, rather than re-setting it each timestep based on the cell centers-of-notes, as this more accurately captures sharp divergences in collisions so gives more accurate bouncing as opposed to artificial sticking. However to prevent the estimates from unphysically diverging, the integrated volume is related back to the direct kernel volume decomposition if the two diverge too severely, on a timescale equal to a few times the slowest strain-wave or sound-wave crossing time of the local density gradient (to prevent accumulated integration errors in very long-duration integrations from driving noise).

Impact Cratering with Damage and Porosity

Basalt Cratering (test/jutzi_crater): A 2D basalt projectile striking a porous basalt target, with the analytic Tillotson basalt preset (EOS_TILLOTSON, EOS_ELASTIC) and the full damage and porosity model (EOS_DAMAGE_POROSITY=7: the Jutzi et al. 2008 P-\(\alpha\) compaction model with Grady-Kipp fracture and a Drucker-Prager yield surface). The IC is generated locally by make_ics.py. The test requires the run to reach its end time with mass conserved to 0.1%, the Damage, Distention and ActiveCracks fields to be written, at least one cell to be damaged (the shock triggered fracture), and at least one cell to be compacted below the initial distention \(\alpha_{0}=1.275\) (the shock pressure exceeded the crush threshold). It is a functional check of the damage and porosity model; it does not compare crater profiles against Jutzi et al. 2008.

P-alpha Crush-Curve Unit Test (test/jutzi_crush_unit): A standalone unit test of the porous-compaction (P-\(\alpha\)) crush curve in solids/jutzi_crush_curve.h (Jutzi et al. 2008, Eq. 8), using a small C++ harness with no GIZMO build or MPI. Over a sweep of pressures and porous, icy and non-porous material parameters, the harness must match a Python reference implementation bit for bit and reproduce analytic basalt values. The curve must also be monotonically non-increasing, equal to \(\alpha_{0}\) below the elastic limit, and equal to 1 above full compaction. It complements the test/jutzi_crater cratering problem above.

Grain-to-Solid Promotion

Grain-to-Solid Promotion (test/grain_promotion): A smoke test of GRAIN_FLUID_PROMOTION. In a 2D box of Tillotson elastic-solid gas cells (EOS_TILLOTSON, EOS_ELASTIC) with a back-reacting grain fluid (GRAIN_FLUID, GRAIN_BACKREACTION), one grain super-particle above the promotion mass threshold must be converted into a solid gas-type cell. The test checks that the grain count drops and the gas count rises by the same number. The new cell must carry the olivine Tillotson composition, and total gas plus grain mass must be conserved to 0.1%.

Self-Gravitating Planetary Bodies

Both problems start from ICs built with the hydrostatic-equilibrium IC builder described in Building Self-Gravitating Body ICs.

Layered Planet in Hydrostatic Equilibrium (test/hse_earth_smoke): An approximately Earth-mass Tillotson body (olivine mantle and iron core, CGS units) built in hydrostatic equilibrium and run for a few dynamical times (\(t_{\rm dyn}\sim 810\) s). The body should simply sit there: if the IC and the solid EOS agree, the rms velocity in units of \(v_{\rm dyn}\) stays small and the density profile does not drift. analyze_hse.py reports, per snapshot, the mean and rms \(|v|/v_{\rm dyn}\), the radial density profile against the IC, the centre-of-mass drift, and the largest excursion of any particle from its initial radius. Variants (each one IC script plus one params file): a single-material basalt body; Fibonacci-shell placement instead of glass relaxation; the same with a precomputed kernel radius in the IC (INPUT_READ_KERNELRADIUS); and self-gravity off, which separates EOS and pressure errors from gravity errors. It is run by hand (see its README), not by pytest.

ANEOS Giant Impact (test/aneos_giant_impact): Evolves a differentiated uniform-density sphere with real M-ANEOS forsterite (mantle) and iron (core) tables from Stewart et al. (2019, 2020). Tests that the code can load real ANEOS tables, handle multi-material particles, and run without crashing. Downloads tables from Zenodo and converts them automatically. Uses EOS_ANEOS, HYDRO_MESHLESS_FINITE_MASS.

Tabulated Equations of State

ANEOS Unit Test (test/aneos_unit): Standalone unit test for the ANEOS module. Generates an ideal-gas SESAME table, loads it, and verifies table reading, bilinear interpolation, temperature inversion (Newton-Raphson), derived quantities (Cv, Gruneisen parameter), and boundary clamping. Does not require running a GIZMO simulation.

ANEOS Shock Tube (test/aneos_shocktube): A standard Sod shock tube using the ANEOS tabulated EOS with a gamma=1.4 ideal-gas table. Validates the full ANEOS pipeline (table loading, T-inversion, EOS dispatch, hydro coupling) by comparing against the exact Riemann solution. Uses EOS_ANEOS, HYDRO_MESHLESS_FINITE_MASS, BOX_SPATIAL_DIMENSION=1.

Giant-Planet H/He EOS (test/cd21_hhe_compression): A 1D periodic box of hydrogen-helium gas (\(Y=0.275\)) with the Chabrier & Debras (2021) equation of state, read through the EOS_ANEOS table path after conversion to SESAME format by initial_conditions/eos_tools/cms_to_sesame.py. The IC (128 cells at \(\rho=10^{-2}\) g cm${–3}$, \(T=10^{3}\) K) carries a single-mode velocity perturbation that drives a small adiabatic compression wave for about ten sound-crossing times. The test checks that the run completes with finite, positive density and internal energy. It is a functional check of the table path for substellar and giant-planet interiors; it does not compare against Jupiter or Saturn interior models.

Dust-Gas / Particle-Laden Dynamics

A number of different studies have now used the dust-particulate dynamics modules of the code (including e.g. Epstein or Stokes or Coulomb drag on dust or aerodynamic particles, Lorentz forces on charged grains, and more). With the release of some of the methods papers for this, the dust/particulate modules are public. I have therefore added some simple test problems for validation and to acquaint users with the modules. The initial conditions, example parameter files, and list of compile-time (Config.sh) flags which must be set to run each problem are publicly available at the same location as the rest of the files above. Users are encouraged to expand the set of tests and implementations of additional physics (e.g. collisions).

Uniform Acceleration-Deceleration of a Dusty Box

This is a variant of the ‘dustybox’ test, a simple test problem which was popularized by Guillaume and Price, 2011, MNRAS, 418, 1491. We initialize a homogeneous periodic box of unit size, full of gas with an idealized adiabatic equation-of-state with unit density and sound speed and zero velocity, and uniform unit density of dust/particulates with initial velocity \(v=1\) in some direction. The two are coupled via some drag law, which decelerates the dust while the ‘backreaction’ from the dust accelerates the gas, until the mixture reaches equilibrium with both components moving at \(v=1/2\). If the drag coefficient (\(=1/t_{s}\)) is independent of the relative velocity of the mixture, then the exact solution is trivial, with the gas moving at \(v_{g}=(1-\exp(-t/2 t_{s}))/2\) at all times \(t\).

This is a simple test used to validate the ‘backreaction’ term and total momentum conservation. It can also be trivially run in any number of dimensions, or for any varied dust-to-gas ratio or drag coefficient. In the default dust implementation in the code, the default drag law is not a constant drag coefficient, but actually Epstein drag. This makes the exact solutions slightly more complicated, but it is still easy to solve for the difference between the dust and gas velocities \(v_{d}-v_{g}=2\,\psi/(1-\alpha\,\psi^{2})\) where \(\psi\equiv (1+\sqrt{1+\alpha})^{-1}\,\exp(-2\,t)\) and \(\alpha=15\pi /128\), for the default parameters here.

Initial conditions: “dustybox_ics.hdf5”

Parameterfile: “dustboxwave.params”
(Be sure in the parameterfile to choose the correct initial condition file)

In Config.sh, enable: 

BOX_PERIODIC  
BOX_SPATIAL_DIMENSION=1  
SELFGRAVITY_OFF  
GRAIN_FLUID  
GRAIN_BACKREACTION  
EOS_GAMMA=(5./3.)  
EOS_ENFORCE_ADIABAT=(3./5.)

The default test is 1D just for simplicity. You can trivially make a 2D or 3D test initializing a box with the included IC-making python script in GIZMO. Feel free to vary the equation of state of the gas, drag coefficient (varying Grain_Size_Min and Grain_Size_Max in the code - these are equal so all grains have the same size), or dust-to-gas ratio (mass of grains in the ICs). You can even manually edit the code lines in grain_physics.c to insert different drag laws.

A Damped, Two-Fluid (Coupled Dust/Particulate-Gas) Wave

This is a variant of the ‘dustwave’ test from Guillaume and Price, 2011, MNRAS, 418, 1491. This uses the same setup and nearly-identical ICs to the ‘dustybox’ test. In fact the parameterfile is identical, you just need to swap out the IC file.

The difference is, instead of initializing the dust with a bulk velocity, we initialize a periodic linear wave in the dust and gas, with amplitude of \(\sim 10^{-4}\) in code units. In the strictly-linear regime, it is straightforward (but tedious) to solve for the eigenfunctions of these waves, for a given dust-to-gas ratio, mean drag coefficient, functional form of the drag law, dust-to-gas ratio, etc. However, particularly for the Epstein drag case adopted here, the functions do not have clean closed-form expressions. So instead we provide a table with an exact numerically-tabulated solution for the default setup here.

This is a much more sensitive test of the dust-gas coupling and back-reaction terms. Like the test above, it can be run in any number of dimensions, or any drag law or dust-to-gas-ratio or coupling coefficient. Because the flow is smooth, it can be used to test numerical convergence rates. Because the wave amplitude is weak (\(\sim 10^{-4}\)), even very small errors will corrupt its evolution. And the propagation of the wave in both dust and gas depends sensitively on accurate back-and-forth coupling between the two media.

Note that in this particular problem, with zero forces on dust or gas and a homogeneous background, the waves are always damped. But as shown in Squire and Hopkins 2018, ApJL, 856, L15 (arXiv:1706.05020), if there is any difference in non-drag forces acting on the dust and gas (e.g. if we introduce an external force which acts even slightly differently on the dust and/or the gas, or a pressure gradient in the gas, or Lorentz forces on the dust), the system becomes unstable to a family of ‘resonant drag instabilities’ which rapidly amplify into strongly non-linear behavior of the dust and gas. This test problem can be used as a starting point to explore these instabilities as well.

Initial conditions: “dustywave_ics.hdf5”

Parameterfile: “dustboxwave.params” (pytest: test/dustywave)
(Be sure in the parameterfile to choose the correct initial condition file)

In Config.sh, enable: 

BOX_PERIODIC  
BOX_SPATIAL_DIMENSION=1  
SELFGRAVITY_OFF  
GRAIN_FLUID  
GRAIN_BACKREACTION  
EOS_GAMMA=(5./3.)  
EOS_ENFORCE_ADIABAT=(3./5.)

The default test is 1D just for simplicity. You can trivially make a 2D or 3D test initializing a box with the included IC-making python script in GIZMO. Feel free to vary the equation of state of the gas, drag coefficient (varying Grain_Size_Min and Grain_Size_Max in the code - these are equal so all grains have the same size), or dust-to-gas ratio (mass of grains in the ICs). You can even manually edit the code lines in grain_physics.c to insert different drag laws.

The exact solutions for the default setup are provided in the file “dustwave_exact.txt”. This gives the solution for the default parameterfile values, at time =1.2 (in code units). The three columns are: (1) x-coordinate position, (2) value of x-velocity of dust, (3) value of x-velocity of gas.

Cooling, Chemistry, and Two-Temperature Tests

GMC Cooling (test/gmc_cooling): Evolves an idealized giant molecular cloud with full cooling and chemistry (COOLING, COOL_LOW_TEMPERATURES, COOL_METAL_LINES_BY_SPECIES, METALS). Validates the density-temperature relation produced by the cooling physics against a reference solution.

GMC Cooling with Radiation Transfer (test/gmc_cooling_rt): As above, but with radiation transport enabled (RT_EVOLVE_INTENSITIES or LEBRON), testing the combined cooling+RT pipeline.

Two-Temperature Plasma (test/two_temperature): Runs the test/gmc_cooling cloud (it reuses that test’s IC, so run gmc_cooling once first) with TWO_TEMPERATURE_PLASMA=1, evolving an electron temperature \(T_{e}\) separately from the gas temperature, under SINGLE_STAR_STARFORGE_DEFAULTS + MAGNETIC + COOLING + METALS. The electrons start out of equilibrium at \(T_{e} = 0.1\,T_{\rm gas}\). The test checks that this seed is applied at \(t=0\) (median ratio \(0.1 \pm 0.005\)), and that Spitzer electron-ion equilibration has acted by the final snapshot (median \(T_{e}/T_{\rm gas} > 0.5\); at these densities the analytic relaxation time is far shorter than one timestep). Both temperatures must also stay finite and within a physical range.

Star Formation and Single-Star Feedback Tests

These problems exercise the STARFORGE-type individual-star physics (SINGLE_STAR_STARFORGE_DEFAULTS): sink formation and accretion, and the radiative, wind, jet and supernova feedback of individual stars.

Shu 1977 Collapse (test/shu1977): The classic isothermal sphere collapse solution from Shu (1977, ApJ, 214, 488). Tests self-gravitating isothermal gas dynamics and is a standard benchmark for star formation simulations. A rotating version of this collapse with protostellar jets is test/shu_jets (below).

Rotating Shu Collapse with Jets (test/shu_jets): The test/shu1977 singular isothermal sphere, spun up to solid-body rotation with \(E_{\rm rot}/|E_{\rm grav}| = 0.1\) and run with kinematic protostellar jets (SINGLE_STAR_FB_JETS, JET_DIRECTION_FIXED_Z), with jet cells spawned at 0.1 of the ambient gas-cell mass. Each run must form exactly one sink, conserve gas plus sink mass to \(10^{-6}\), keep the sink mass within a broad window, and launch an outward outflow of correctly sized spawned cells. The outflow must also be collimated along \(z\), both in position and in the principal axis of its kinetic-energy tensor. Four runs cross the default spawned-cell merging against SINK_SPAWN_NO_MERGE with COOLING against COOLING + SINGLE_STAR_FB_RAD. A separate check (test_shu_jets_merging_energetics) requires the merging run to retain more than half the no-merging run’s gas energy, outward momentum and sink mass at matched times.

Massive Protostellar Core (test/shu_M120): Collapse of a \(120\,M_{\odot}\) protostellar core with an \(r^{-2}\) density profile, under SINGLE_STAR_STARFORGE_DEFAULTS + COOLING + SINGLE_STAR_FB_RAD. It should produce a single sink with an approximately constant, high accretion rate, and smooth profiles of gas, dust and radiation temperature. The binned radial profiles of the three temperatures and the sink mass are compared against a reference solution (shu_M120_exact.hdf5, downloaded automatically) to 10%. It is parametrized over the same baseline / TRANSPORT_SUBCYCLE=10 / TRANSPORT_SUBCYCLE=10 + TRANSPORT_SUBCYCLE_COOLING variants as test/HII_region.

HII Region (test/HII_region): A \(30\,M_{\odot}\) zero-age main-sequence star sits in a uniform, optically thick \(20\,{\rm pc}\) box of mean density \(100\,{\rm cm^{-3}}\) (gravity off), and its multi-band radiation (SINGLE_STAR_RT_DEFAULTS, RT_CHEM_PHOTOION=1, NUV, optical/NIR, IR and photoelectric bands, reduced speed of light) must ionize a Strömgren-type HII region. The test requires the peak of the binned median gas temperature to lie at the photoionization-equilibrium value (7000–11000 K), which catches an over-hard spectrum, missing nebular cooling or spurious over-cooling. It is parametrized over a baseline, TRANSPORT_SUBCYCLE=10 (subcycled radiation transport), and TRANSPORT_SUBCYCLE=10 + TRANSPORT_SUBCYCLE_COOLING. The subcycled runs must reproduce the baseline’s radial profiles of ionized fraction and NUV and optical/NIR radiation energy density to 10%. Plots of the ionization-front radius versus time are also produced.

Single-Star Supernova (test/SN_singlestar): A \(10\,M_{\odot}\) star near the end of its life, at the center of a \(5\times10^{4}\,M_{\odot}\), \(n_{\rm H}=100\,{\rm cm^{-3}}\) periodic box, explodes as a \(10^{51}\) erg supernova on the first timestep (SINGLE_STAR_FB_SNE, gravity off). It is parametrized over adiabatic, COOLING, and COOLING + SINGLE_STAR_FB_RAD variants. In all three, the density peak at early time must lie within 30% of the analytic Sedov-Taylor shock radius and be moving outward. The adiabatic run must also show a strong-shock density jump (more than twice ambient) and conserve the injected energy to 10%. A separate check on the cooling variant’s output (test_SN_thermo_fidelity) bounds the temperature-versus-internal-energy relation of the warm molecular shell and the remnant’s total thermal energy against calibrated reference values, which guards the internal-energy-to-temperature inversion in the cooling module.

Single-Star Wind Bubble (test/wind_singlestar): A \(100\,M_{\odot}\) ZAMS star (SINGLE_STAR_FB_WINDS=2, \(\dot{M}=10^{-4}\,M_{\odot}\,{\rm yr^{-1}}\), \(v_{w}=3000\,{\rm km\,s^{-1}}\)) blows a bubble into the same uniform box as test/SN_singlestar, compared against the Weaver et al. (1977) similarity solution \(R_{2} = \alpha\,(L_{w}t^{3}/\rho_{0})^{1/5}\). It is parametrized over the wind injection mode (spawned discrete wind cells versus local mechanical injection) and over adiabatic, COOLING, and COOLING + SINGLE_STAR_FB_RAD thermodynamics. The adiabatic runs must reproduce energy conservation (\(E_{\rm kin}+E_{\rm th}=L_{w}t\); 10% for spawning, looser for local injection), the \(R\propto t^{3/5}\) growth law, a time-constant similarity coefficient, and \(\alpha=0.88\) to 10%. The radiative runs must match \(\alpha=0.76\) to 10%. Because the exact solution has a contact discontinuity that should never mix, the test is also a sensitive check on the merging criteria for spawned cells; the local-injection variants are currently non-strict expected failures (they over-deliver energy by roughly 25–30%; see the README).

Galaxy-Formation and Feedback Tests

Most of these are short runs of the isolated galactic disk (isodisk_ics.hdf5) or of a FIRE galaxy. They check that each feedback or source-injection module is active and conserves mass, rather than comparing against an exact solution.

Isolated Disk: Mechanical Feedback (test/isodisk_mechfb): The isolated disk with star formation and GALSF_FB_MECHANICAL (SNe and stellar mass loss) on a COOLING + GALSF + METALS base. Pre-existing old star particles produce supernovae from the first timestep. The test checks that the run reaches its end time with total baryonic mass (gas plus all stellar types) conserved to 1%. The directory also holds Config variants that must be built and run by hand (pytest runs only the default): two cosmic-ray variants (Config_cr.sh and Config_cr0.sh: FIRE_PHYSICS_DEFAULTS=3 with FIRE_CRS=-1 or 0 and FIRE_MHD), and an explicit-radiation variant (Config_rt.sh, isodisk_mechfb_rt.params: RT_M1 with the stars as sources in the NUV, optical/NIR, IR, photoelectric and photoionizing bands). See the README there.

Isolated Disk: Thermal Feedback (test/isodisk_thermalfb): As test/isodisk_mechfb, but with GALSF_FB_THERMAL (pure thermal SN energy injection) in place of mechanical feedback. The test checks completion and conservation of total gas plus star plus black-hole mass to 0.1%.

Isolated Disk: Black-Hole Sinks (test/isodisk_mechfb_sinks): The isolated disk (with its central black hole) under FIRE_PHYSICS_DEFAULTS=3 + FIRE_BHS, which enables sink accretion and swallowing. The test checks that the run completes and that the black hole’s subgrid mass (Sink_Mass) grows by accretion. Baryonic mass, including the sink, must be conserved to 1%. It borrows its IC and cooling tables from test/isodisk_mechfb.

Isolated Disk: Dark-Matter Velocity Dispersion (test/isodisk_dmdispersion): The isolated disk with the Springel & Hernquist effective-EOS model and velocity-scaled subgrid winds (GALSF_EFFECTIVE_EQS, GALSF_SUBGRID_WINDS, GALSF_SUBGRID_WIND_SCALING=2), which need the local dark-matter velocity dispersion at every gas cell. The test checks that the dispersion computation ran (its status line appears in stdout) and that most gas cells carry a non-zero value. Gas plus star mass must be conserved to 0.1%. The extra Config_dm_heating*.sh and *_dmheating_*.params files (DM_HEATING) are for manual runs and are not used by pytest.

FIRE Cosmological Galaxy (test/fire): A short cosmological run of a downsampled (10%, with particle masses raised 10x to conserve total mass) m11i-like dwarf galaxy from \(a=0.2564\) (\(z\approx 2.9\)) to \(a=0.257\), with the full FIRE physics set (FIRE_PHYSICS_DEFAULTS=3, FIRE_BHS) in a periodic box with PMGRID=512; it starts from a snapshot (RestartFlag=2). It is parametrized over a baseline and a tidal-timestep variant (TIDAL_TIMESTEP_CRITERION, ADAPTIVE_TREEFORCE_UPDATE=0.06). The test requires dark-matter (and, if present, type–2) mass to be conserved exactly, and the final gas, star and black-hole masses to agree with a reference snapshot to 10%; it also writes the gas density-temperature distribution and density histogram for inspection.

FIRE Galaxy: Radiation Sources (test/fire_rtsources): A short run of the FIRE m11i dwarf-galaxy IC (z ~ 2.9) with FIRE_PHYSICS_DEFAULTS=3 + FIRE_BHS and explicit M1 radiation transport (RT_M1, RT_COMOVING, RT_SOURCES=48, with NUV, optical/NIR, IR, photoelectric and photoionizing bands). Stars and black holes inject radiation from the first step. The test checks that dark-matter mass is exactly conserved and that the gas carries non-zero radiation energy after injection; it is a completion and activation check, not an accuracy test.

Forged-in-FIRE Nuclear AGN Disk (test/forgedinfire): A short restart (RestartFlag=2) from a downsampled snapshot of a nuclear-scale AGN accretion-disk zoom (about \(10^{6}\) gas cells, a black hole and stars). It uses full radiation-MHD, dust temperatures and star formation in the hybrid single-star/SSP nuclear-zoom model (SINGLE_STAR_AND_SSP_HYBRID_MODEL_DEFAULTS, SINGLE_STAR_AND_SSP_NUCLEAR_ZOOM, USE_TIMESTEP_DILATION_FOR_ZOOMS). Primarily a does-it-run test of that pipeline, it produces radial profiles and phase diagrams for inspection. If the reference snapshot forgedinfire_exact.hdf5 is present, the final total gas and star masses must agree with it to 10%. The IC is about 260 MB and gzip-compressed, so it needs an HDF5 build with the deflate filter.

Nuclear Burning and Stellar Equation-of-State Tests

Nuclear Network Unit Test (test/nuclear_unit): Standalone unit test for the NUCLEAR_NETWORK module. Exercises the aprox13 alpha-chain solver, optional Coulomb screening, and the tabulated NSE branch over a grid of thermodynamic states without running a full hydro simulation. Does not require running a GIZMO simulation.

Helmholtz EOS Unit Test (test/helmholtz_unit): A standalone unit test of the EOS_HELMHOLTZ module, which supplies the electron-positron and radiation equation of state for the burning tests in this subsection. It compiles a small C++ driver against src/eos/helmholtz/helmholtz.cc and evaluates the EOS over a grid of \((\rho, T, \bar{A}, Y_{e})\), using helm_table.dat (downloaded if absent). The driver checks the analytic ion and radiation terms, round-trip temperature inversion, the thermodynamic consistency (Maxwell) relations, monotonicity in temperature, and physical limits (sound speed below \(c\)); the test fails if any check fails. Does not require running a GIZMO simulation.

Nuclear Detonation (test/nuclear_detonation): A 3D-elongated 1D detonation test (BOX_LONG_X=4) for thermonuclear flame propagation in a degenerate-CO mixture, exercising the coupled EOS_HELMHOLTZ + NUCLEAR_NETWORK + NUCLEAR_NETWORK_NSE_TABLE pipeline. Tests that a self-sustained burning front advances at the expected speed and produces the expected NSE-equilibrated post-shock composition.

X-Ray Burst (XRB) (test/nuclear_xrb): A 1D test problem for thermonuclear burning in a neutron-star accretion layer (Type I X-ray burst regime), with EOS_HELMHOLTZ + NUCLEAR_NETWORK + NUCLEAR_NETWORK_NSE_TABLE + METALS. Exercises the hot, degenerate-electron regime of the EOS and the network’s stability under high temperatures and densities.

Numerical Stress, I/O, and Infrastructure Tests

Poisson Box (test/poisson_box): A periodic 100 pc box of magnetized, cooling gas (MAGNETIC, MHD_B_SET_IN_PARAMS, COOLING, METALS, COOL_METAL_LINES_BY_SPECIES) at \(T\sim10^{4}\) K and mean \(n_{\rm H}=1\) cm${–3}$, whose cells sit at uniformly random (Poisson) positions – no lattice and no glass. A random point set is the least regular arrangement the neighbour search, the kernel-radius iteration and the gradient and MHD estimators can be handed: local cell counts fluctuate by order unity on the kernel scale, so every kernel radius must converge from a poor initial guess. Gravity is in the code path but negligible (GravityConstantInternal\(=10^{-100}\)). The test runs a few all-active steps on \(50^{3}\) cells and requires: the final time to be reached; gas mass conserved to round-off; finite, positive densities and kernel radii; and the magnetic energy of the uniform seed field to stay constant to \(3\times10^{-4}\). make_poisson_box_ics.py writes the IC at any resolution and mean density (e.g. python make_poisson_box_ics.py 30 50 80 100), which makes the problem a convenient scaling benchmark as well (see the CPU-GPU performance table in Performance-Portable Heterogeneous Execution).

Binary IC Reading (test/read_ic_binary): A regression test that GIZMO can start from a GADGET-format binary IC (ICFormat 1/2) without a Temperature block, even when temperature output (OUTPUT_TEMPERATURE, implicit for most COOLING runs) is enabled. The 512-cell IC is generated locally. The default variant must read the IC, run, and still write Temperature to its snapshots. A sensitivity variant built with INPUT_READ_TEMPERATURE must instead fail with the end-of-file read error the original bug produced, which confirms that the IC still reproduces the condition being guarded.

C++ Unit Tests (test/unit): Standalone C++ tests of low-level code utilities (currently the Vec3 math type: construction, element access, iteration, arithmetic operators, norms), compiled and run directly by pytest without a GIZMO build or MPI. New test_*.cc files in the directory are discovered automatically. (test/trivial holds a single always-passing test that checks the pytest installation itself.)

Compile Suite (test/compile_suite): Not a physics test; systematically compiles GIZMO with many different combinations of Config flags to check for compilation errors and flag incompatibilities.

12. Useful Additional Resources

This section addresses some common questions and provides some additional resources for new users, which are not part of GIZMO itself.

Visualization, Radiative Transfer, and Plotting

Many questions I get about GIZMO are actually questions about how to visualize and plot data. GIZMO doesnt do this itself, of course, but there are many public codes out there which are compatible with GIZMO and have excellent tools available for visualization. Remember, anything that says it is compatible with “GADGET” formats is compatible with GIZMO outputs. Just a few examples include (thanks to Robyn Sanderson for suggesting several here):

For more detailed radiative transfer performed on the simulations, there are many popular tools again.

Halo/Group-Finding and Structure Identification (Post-Processing)

Many people ask about automated tools for halo-finding or structure identification in the simulations. This is a quite large and mature industry onto itself, on which many many papers have been written. I’ll just note a couple popular codes that people commonly use to process GIZMO outputs:

Other Analysis Tools

While there are a huge number of analysis tools, and users can of course write analysis software in any language they like, I highly recommend Python for analysis. It’s open source, supported by the national centers (so you can run remotely), and much astronomy code development is based on it.

General Super-Computing Questions

Questions about computing in general should be directed at the help resources for whatever computers you are using to run GIZMO. This includes things like “I can’t get FFTW to compile,” and “I can’t get GIZMO to compile” (unless its a specific compiler error internal to the GIZMO code). But to help get started on submitting jobs, here’s some useful resources for users not experienced with these systems.

To submit a simulation job, first read up on the machine User Guide (and of course, I’m assuming you have read the code User Guide cover-to-cover). Then compile the code, and put the compiled code (should compile to “GIZMO”) in the intended run-time directory with the run-time parameters file (which we will call here “params.txt”). Then submit the job by submitting the relevant script.

On Frontera (TACC), a typical SLURM script looks like:

#!/bin/sh  
#SBATCH -J insert_job_name_here  
#SBATCH -p normal  
#SBATCH -t 48:00:00  
#SBATCH -N 16 -n 896  
#SBATCH -A AST12345

#export OMP_NUM_THREADS=2  
ibrun ./GIZMO ./params.txt 1>gizmo.out 2>gizmo.err

Where -p specifies the queue (default=normal, short runs=development), -t specifies the wall-clock runtime (hours:minutes:seconds), -N specifies the node number, -n the total number of MPI processes. Frontera has 56 cores per node, so here we request 16 nodes with a total of 896 MPI processes (up to n=N*56). -A is the account to charge it to. When you log in, the first thing you see should be a list of the accounts you are listed on. If there is only one, you dont need to specify this (it will default to that one). Other SLURM-based machines will use a very similar script structure; consult the user guide for your specific machine for queue names, core counts, and launcher commands.

The line export OMP_NUM_THREADS should have the leading # removed if and only if you compile the code in hybrid OPENMP-MPI mode. This sets the number of OPENMP threads. In the Config.sh file used to compile the code, the value of OPENMP must be set/uncommented (e.g. OPENMP=2, here) and MUST EXACTLY MATCH the integer value here. Also, since there are now OMP_NUM_THREADS number of threads per MPI process, the -n above (number of MPI processes) must be lowered. Each thread uses one CPU. So the maximum value of -n is -N * (cores per node) / OMP_NUM_THREADS.

Finally the last line submits the job, tells it the executable (GIZMO, in the same folder as the job is submitted from), the parameterfile GIZMO should use, and the names of the standard output and error files to write to.

This should be saved in its own file, with NO LEADING WHITESPACE before each line. Let’s call the file “run.bsub”. The run is then submitted into the queue with the command “sbatch run.bsub”.

Different machines will use different submission scripts, and may have different notation entirely for the commands above, but usually the set of values you must specify is very similar (its just the names that change) – the user guides for the different machines tell you what their naming conventions are.

And remember, once you get your simulations running, ALWAYS BACK UP YOUR DATA (whatever you’re doing, it’s just good practice)!

13. Frequently Asked Questions

This section addresses some of the most-commonly asked questions from new users.

Where to Go for Help?

Good news, you’re in the right spot: first, read this guide. Please read it completely before asking for help: the document is long precisely because it already contains answers to more than 99% of all questions I receive from users. Make sure you also check the Template_Config.sh file for the relevant modules/flags, and even take a look at the source code to see if there are obvious notes (often the code has comments to clarify things for users) or definitions.

If that really doesn’t answer your question, please check our Issues and/or Discussions fora at the GIZMO GitHub site here. There are various posts archived there, with code issues, bugs and their fixes, feature requests and their modules that were developed, all maintained. First check if someone else has already asked the same question, then (if not), please post and join the discussion – that’s what its for, and other users most likely have answers for you. Note that unfortunately, the older Google Groups system we had used had to be closed permanently owing to security and spam issues (and we were not able to import the older posts).

If you still cannot get an answer, the best thing to do is to find another experienced user, and ask them directly. Many such users are active in the development community, and of course you can reach out to colleagues, collaborators, and members of the community who have published many papers using GIZMO. And of course if you can’t find anyone there, you can contact the code developers (Phil or the salient developers of the modules in question, which are listed in detail in the Template_Config.sh file, and User Guide descriptions).

What Are ‘Optimal’ Code Settings? (aka ‘My Run is Slow’)

This is (by a huge margin) the most common question I receive, including variants like ‘What is the expected performance on X problem’, or ‘Why is the code spending so much time on X?’, or ‘How can I improve speed (or reduce load imbalances)?’, or ‘What is the optimal number of CPUs or configuration for this problem?’ or ‘Why does GIZMO take longer than this other code to run this problem?’ etc. Unfortunately, it also has no simple answer: there is no single optimal configuration.

The reason is straightforward: GIZMO is designed to flexibly run a huge range of different types of simulations as specified by the user. With the right choices, GIZMO should be as fast or faster than almost any other codes out there on almost all problems I have seriously tested. But a simulation of linear MHD instabilities (where every element is nearly homogeneous and advancing in lockstep) and a ‘zoom-in’ simulation of galaxy formation (where tiny, dense clumps and star clusters have timesteps a million times shorter than the volume-filling inter-galactic medium) have wildly different optimal parallelization strategies. Likewise, a problem running on a single node with 20 shared-memory cores will have totally different bottlenecks and overheads compared to a problem running on a hundred thousand nodes linked via infiniband. It simply isn’t possible (despite what some misleading compiler libraries may claim in their sales pitches) to write a code that can predict your specific use case and all the bottlenecks or gains that will apply. This is why the very fastest codes are often custom-written for a single extremely specific problem and configuration, even taken to the custom-hardware level (think GRAPE boards for direct-N-body integration, or custom chips for mining cryptocurrency).

This means some parameters will always need to be tailored to your use case and the specific problem being simulated. And to be clear, ‘specific problem’ in this context means a combination of physics enabled, resolution, node configuration and processor number, type of CPU, memory, how deep the timestep hierarchy is, ratio of collisionless to gas elements, and so on: it does not simply mean a broad topic like ‘cosmological simulation.’ Until you have experience running a specific type of simulation on a specific machine, you simply have to experiment with many different choices to identify the optimal configurations (I myself still do this every time I’m using a new cluster).

There are some general ‘rules of thumb’ however, regarding which parameters you should consider, and how they might impact your performance. Be sure, if you are testing, to enable OUTPUT_ADDITIONAL_RUNINFO in your Config.sh file to get more fine-grained info. The best thing you can do is use actual software performance profiling tools to break down the time in every subroutine and call of the code: almost every computing center provides these tools and instructions on how to use them, please talk to the specialists there about how to use these tools!

Compilers: On any machine the choice of compiler (e.g. intel, gnu, pcc), compiler flags (-O1/2/3, custom optimizations), and which MPI libraries you use (e.g. intel-MPI, openMPI, mvapich), is important, but highly machine-specific. Read the user guides specific to your platform, and experiment. If you are running on a machine listed in Makefile.systype, check its description in Makefile – almost all have helpful notes there describing advice, most useful modules, some tests, etc. If you’re running on a new machine, check Makefile for advice for machines with similar hardware (and once you determine best compilers, add it to the code). In general, intel or pcc compilers often provide best performance, with gnu a bit slower. Modern forms of the MPI libraries tend to be comparably fast, it is more about what the computing center has optimized for. Compiler optimization flags require care: usually 95% of the performance gain comes from mild or intermediate optimization like -O1 or -O2. More aggressive optimization should be done only with care: -O3 and other flags can introduce inaccuracy for speed, break the code entirely (by moving things out-of-order), or make things slower (by ‘guessing’ patterns incorrectly). If unsure, scale down the optimization and see if you lose anything (always safer to use lower optimization). Custom flags tend to be minor differences. The choice of which version of other code libraries (GSL, FFTW3, MKL, etc) tend to make a very small difference.

Config.sh Settings: A couple Config settings can have a big impact, you should explore different choices for them. (1) OPENMP: see node configuration options below. (2) DOMAIN_SEGMENTS_SCALE: the code already picks the number of domain segments per task automatically from the problem size and task count, so this only scales that choice. Raising it allows more sub-blocking of domains and so more flexibility in the decomposition, useful for highly inhomogeneous problems (e.g. deep timestep hierarchies), but introduces larger memory overheads and more communication. Generally lower for problems that are memory- or communication-limited, higher for problems which are imbalance-limited. (3) If you’re using PMGRID for gravity (in e.g. cosmological simulations with periodic boxes), changing the value here can optimize by trading time between the FFT algorithms used for the PMGRID and tree (used on scales below PMGRID). Higher values will cost memory and more FFT time and can make gravity less accurate on small scales (as the tree-solver is more accurate there), but reduce tree-walk times and imbalances.

Parameterfile Settings: The params file also has a couple settings that can have a big impact. (1) DomainBuild_ActiveFraction (and its tree-only companion TreeRebuild_ActiveFraction): this requires more hand-tuning from experience than almost any other parameter. See its description: decreasing it will force the code to rebuild the domain decomposition more frequently, and can reduce load imbalances. But making it too frequent will cause it to bottleneck the code. A good ‘rule of thumb’ is that if the code is spending comparable time on imbalances (treeimbal+densimbal+hydimbal in cpu.txt) and domain decomposition (domain in cpu.txt), then this is set about right (but if both of these are very large, this is obviously an issue). (2) PartAllocFactor: this sets how much particle-load imbalance the decomposition may accept in order to reduce work-load imbalance. Raising it does not buy room for data imported from other tasks, which is measured and allocated separately, and it does enlarge the gravity tree’s node arrays, so a large value costs memory without helping. (3) Other parameters like force softening, neighbor number, time between snapshots or statistics, etc, are generally small corrections, unless they are set to inappropriate values and cause problems. For example, setting a force softening much too-small means the timesteps are artificially small; setting it too large will introduce large imbalances as the particles need to perform direct-\(N^{2}\) gravity operations between every neighbor within their force-softening kernels.

Node, CPU, MPI Configuration: This is the trickiest but often most important issue. Again, read carefully the user guides for the machine and consult the IT help staff, and experiment before running production simulations. (1) On some machines, specific thread placement requires some custom options be set: this is especially important if the nodes involve a mix of floating-point and integer cores (on these types of machines, failure to specify the correct custom ‘thread placement’ can lead to order-of-magnitude slower performance, as the threads get placed on overlapping cores, or threads intended for floating-point cores land on integer cores). One example of this is in the Makefile for the ‘BlueWaters’ machine. (2) On some machines, hyper-threading is encouraged to get best performance. Hyper-threads should almost always be OPENMP threads (as opposed to putting 2 MPI tasks on the same CPU). On others hyper-threading will crash. On others, hyper-threading will run but cause an order-of-magnitude slowdown. (3) The balance between the number of OPENMP threads and MPI tasks is highly machine and problem dependent. In setting OPENMP, consider whether you want hyperthreads or want extra memory per task (larger OPENMP). But also note some things cannot be OPENMP parallelized as effectively as MPI parallelized: it can be faster to have more MPI tasks instead of OPENMP threads (for the same core number), especially for smaller jobs. Some small jobs (e.g. couple-hundred-core) may prefer no OPENMP threading at all. As a rule of thumb, larger jobs and jobs with a deeper timestep hierarchy will benefit more from increasing the OPENMP count (depending on the job and computer, I’ve found optimal performance with values of OPENMP ranging from none to 40). But also be aware of the node configuration: not all cores on a node have shared memory, and if you make OPENMP larger than the number with a shared memory access, the performance will drop dramatically. There really is no substitute for experimenting with this value. (4) In terms of the total number of cores or threads, you will also need to experiment, as the behavior of the strong and weak scaling will always be highly problem-dependent. As a rule of thumb, you probably want to aim for something like \(\sim 10^{4} - 10^{7}\) resolution elements ‘per thread’, and for many types of problems with deep timestep hierarchies you will find a ‘sweet spot’ more like \(\sim 10^{5} - 10^{6}\). With too many elements per thread you simply aren’t getting all the parallel benefit (strong scaling) you could (and may run into memory issues). With too few, your threads will not have enough work to do each timestep and most will simply idle while a couple others finish or they all wait for communications.

Good luck!

Are There ‘Best’ (More Accurate) Physics Modules or Solvers?

No. There is no single ‘best’ hydro solver, or gravity solver, or physics module, etc, in terms of accuracy (either numerical or physical). There are choices which are better for certain types of problems: either they are more accurate under some conditions, or they are faster and allow you to run higher-resolution and so be more accurate in that way, or they minimize a certain specific type of error you care about, or they are compatible with other modules that allow you to run the relevant physics. But there is never a single always-best choice, and it is important that you think about what works for your problem. Also make sure you read the discussion of Fluid (Hydro) Solvers.

My Run Won’t Start, What Did I Do Wrong?

Again, this is not something with a generic answer. If your run will not start, its almost never a code problem, but a setup problem. Common causes include: (1) code was compiled incorrectly, (2) cant find the libraries it needs, (3) memory settings (see below) are incorrect and ask for more memory than available, (4) the source code, parameterfile, other needed files (ICs, TREECOOL, etc) are not in the correct directories your run script thinks they should be in, (5) your run script was not written correctly for the machine you are running on, (6) your maximum/minimum timesteps are set much too high or low for the problem, (7) your end time or begin time of the run, or box size, or cosmological parameters are set incorrectly, (8) the initial conditions file is not in the correct format, or has data not in the format assumed (remember parameters like INPUT_IN_DOUBLEPRECISION, etc. are needed for those inputs to be read correctly if they are in those formats), (9) your settings (e.g. node number or multipledomains, or force softenings and smoothing length limits) are set such that the code spends its entire run-time in the ‘overhead’ calculations (reading in ICs, initial iteration to converge to smoothing lengths, etc).

Always check all stdout and stderr and run-time output files (including outputs from the job script system if they include those). If no such outputs exist the problem is almost certainly in your job submission script. If outputs exist, search for the error codes. Outputs in stdout with error codes will be error codes in GIZMO – search for them explicitly in the GIZMO source code (do a grep) to find out where the code exited if the description is unclear. Other error codes are machine or compiler-provided: a simple search online will usually find a more useful description.

How do I Avoid Memory Errors/Crashes?

Assuming runs are set up appropriately, begin smoothly, and there are no hardware problems, then by far the most common crash/error for GIZMO simulations is related to running out of memory. This can manifest in many different ways in detail, but with usually involves the code exiting with an error message like Task=%d: Not enough memory in mymalloc_fullinfo() to allocate %g MB for variable '%s' at %s()/%s/line %d (FreeBytes=%g MB) where the various %g values describe the specific memory location of the error and magnitude of the discrepancy. This is particularly commonly associated with the structure ‘GasGradDataResult’ in this output, because in many runs that will be the single most-demanding memory call of the relevant type.

In September 2019 the parallelism was substantially re-structured to unify different modules and in particular to eliminate this particular type of error, which occurs when a given node is set to receive data from an extremely large number of nodes that need to communicate to it (which can occur in some highly imbalanced problems) in a way it could not predict in the old code. Dealing with this involves a pre-pass to check how much it can receive, and iterating the communication over smaller sub-chunks. This does involve some overhead cost in communication, but this is very small in tests, and worth it for removing one of the most common crash cases. Note that this issue, while much rarer, is not fixed for some particular legacy subroutines coded by older users with some particular custom loops over different grids (instead of our normal cell list), such as the turbulent power spectrum computation: you may still see an error like this, in which case the generic memory advice below will apply.

In either case, there are still occasional memory issues. This occurs because, in order to be as flexible as possible (especially with load-balancing), GIZMO attempts to dynamically and flexibly allocate and de-allocate memory and re-scale it as needed. But that means it may reach a point where it has ‘trapped’ itself and has already allocated too much memory but cannot even allocate the memory needed to re-structure memory itself across different nodes (which requires some ability to allocate communication structures, etc). And some parts of the code contributed by other users may not be as ‘safe’ as core parts of the code. So you may see memory errors involving the code immediately quitting with an error that it was unable to allocate the memory it desired, or one of the endrun(NNN) where NNN is number errors which, when you search for it, has a comment to the effect of: in this case, the buffer is too small to process even a single particle.

These are extremely problem-and-machine sensitive, as you would expect, so there is no single fix if you are running into memory problems, but there are a few things you should always check and experiment with.

In run/machine/node configuration: (1) Be sure you are allocating enough nodes to actually accomodate the full size of your problem. Remember that the active memory is much larger than the size of snapshots (which only output a tiny fraction of the data). In addition to code profiling tools, and the memory table dumps produced occasionally, look at the restartfiles (which are an actual flash of code memory at the time they are written) to get an idea of how much memory you really need, and be sure to scale your runs appropriately. If you have a billion cells, and a lot of physics (e.g. chemistry, etc) which means a huge number of variables per cell, you will need a lot of space! So you may need to increase the number of nodes/tasks requested. (2) Try adjusting the ratio of MPI tasks to OPENMP threads. MPI communication makes complete copies of things every sub-domain needs (e.g. tree structure) so that they can be completely autonomous until you exchange specific information (why it works over infiniband and other networks). But this means there are significant memory overheads to every MPI task, and some of these increase per-core (sometimes non-linearly) with the number of tasks (the tree has more structure, for example). OPENMP, on the other hand, only works with physical shared memory, so within an MPI domain only certain cores on a sub-set of a node can be OPENMP threaded. But since it assumes shared memory, it creates effectively zero memory overheads. So taking the same number of cores, you can double the memory per MPI task by reducing the task number by a factor of two and doubling OPENMP. Experiment with the OPENMP values: sometimes we will use larger values just for memory reasons. (3) Of course, if you can find machines or nodes with more memory per core, great!

In the Config.sh file: (1) (ALLOW_IMBALANCED_GASPARTICLELOAD used to appear here; it is retired and has no effect, because the gas-cell capacity now always follows the particle capacity.) (2) There are some algorithms that can be especially memory-intensive. Setting PMGRID or TURB_DRIVING_SPECTRUMGRID to very large values means you need to allocate memory for that size cubed because you are making a three-dimensional grid, so try lowering these values, or looking for other areas where you are asking for large memory overheads. The code dumps of the memory footprint on startup and at particular use times are helpful for this. (3) DOMAIN_SEGMENTS_SCALE multiplies the number of domain segments each MPI task is given. While more segments allow for more sub-chunking, each one carries fixed memory overheads (storing tree and communication data) as well as additional comm-buffers. This is also how you can often overload the comm buffers, since even on a modest number of nodes the number of simultaneous communication requests is your MPI task count times the segments per task (a very common source of problems when set too large). (4) Adjust OPENMP as described above.

In the parameterfile: (1) The working memory pool and the communication chunk (WorkingMemoryPoolSize, CommChunkSize) are worked out by the code and should not normally be set. The code prints the pool size it chose at start-up. If a run stops with a message naming the pool, something needed more of it than was predicted, and setting that parameter is the right response; note that the pool is only one of several places memory is used, so it is not a cap on the task’s total. (2) PartAllocFactor sets the largest particle-load imbalance the decomposition may produce (if the average task holds N elements and this is 5, one task may be given up to 5N). Lowering it reduces the worst-case memory on any one task, at the cost of a worse work-load balance. Raising it does not buy room for imported data – that is measured separately and added on top – and it does enlarge the tree’s node arrays, so raising it to fix a memory problem generally makes the problem worse. If the value is too low the decomposition may fail to find any valid solution at all. (3) (4) Decreasing DomainBuild_ActiveFraction, making domain decompositions more frequent, will allow finer-grained refactoring of the memory balancing, making it easier to avoid memory overflows (at obvious computational cost). Optimal values are wildly problem-dependent, anything from 1e–8 to 0.1 in simulations I have run personally.

My Large Simulation Hangs, but Smaller Runs Work?

If small and intermediate jobs are running, but larger jobs fail immediately, there are a few things to check. First make sure its not a memory issue (larger jobs require more memory, even per particle, owing to larger overheads): see the question above. Second, make sure it isn’t a machine issue (some versions of MPI and some machines require very special commands to run on a very large number of nodes) – check with your IT help.

In older versions of GIZMO, there were also a couple of Config.sh parameters which could be required for large runs (LONGIDS and NO_ISEND_IRECV_IN_DOMAIN). In the current version of GIZMO, these are always on and hard-coded into the Makefile and source, so users do not need to set them.

Why Did Everything Drop to TimeBin=1?

This is a peculiar manifestation of a certain class of errors. In the stdout file, where it shows the number of particles in a given timebin, suddenly (in one or a few timesteps), a huge number of cells end up in bin= 1. This is the smallest allowed timebin. First check that your minimum timestep is not too large (i.e. that this isn’t just the code doing what it should do, but you aren’t allowing it to use small enough timebins).

Assuming this is a real issue, what has usually happened is that somewhere (usually in a neighbor interaction involving terms in a more experimental module), the code divided by zero (or nearly-zero) or otherwise inserted a nan or extremely large or small number. This can happen under some circumstances when un-addressed memory is called (instead of the code just immediately crashing). This assigns a huge value to something like acceleration or velocity or energy, which drops the timebin down. This might only occur in one interaction, but that infinity or nan then propagates through the grid rapidly as elements interact.

Look carefully at the stdout, and profiling info, to diagnose in which subroutine this occurred. Turn on STOP_WHEN_BELOW_MINTIMESTEP to get additional info on the cells when the timestep drops, to see what is jumping to NaN or very strange values. Turn on and off different physics modules in turn while re-starting from a snapshot or restartfile before the error to isolate which modules and sub-routines cause the error.

In my experience, this is usually caused by users setting inappropriate flags (flags not designed for the problem they are simulating, or mutually-incompatible flags), or having a units mistake in their ICs or parameter file (so the timesteps are wildly different from what the user ‘thinks’ they should be), or reading ICs with insufficient precision (so particles have identical positions at floating-point accuracy, solved by setting the input and output Config.sh flags to DOUBLEPRECISION). But obviously, if you think you’ve identified a bug, you should go to the user group and report it.

Are there Public ICs? Analysis Tools? Image/Movie-Makers?

Yes, lots of them! See the sections of this User Guide on Initial Conditions (Making & Reading Them) and Useful Additional Resources.

What Does this Variable Mean?

Read the Snapshot & Initial Condition Files section, as well as the description of the relevant Config.sh flags you have enabled in Config.sh (Setting compile-time options), where most of the outputs are described. If you cannot find it, search the source code, in io.c you can find the map between the HDF5 variable name and what is actually written out.

Example: if you are using radiation-hydrodynamics, and see the variable PhotonEnergy in your HDF5 file, search this in io.c to see it is associated with the case IO_RADGAMMA (the switch for whether or not this variable is written out to a file), so search that and you see it is writing out a vector with N_RT_FREQ_BINS entries per particle, representing the different radiation frequency bins followed in the code in the same order, and writing out the variable CellP[pindex].Rad_E_gamma[k] – the absolute energy of the radiation field (in code units) at frequency k associated with gas resolution element pindex.

What are the Code Units?

See the section of this User Guide on Units. If you aren’t sure whether something is in one unit or another (e.g. is this an absolute energy associated with a particle, or a specific energy, or an energy density?) check the units for sanity, and search for the variable in the source code to see where it is defined and written out.

Can I use this Module? What should I cite?

See the Code Use, Authorship, Citation, Sharing, & Development Rules.

Can the Code do ‘X’?

Read this User Guide to find out, especially the Feature (Physics Module) Set and various subs-sections on all the huge variety of different options in Config.sh (Setting compile-time options).

14. Disclaimer

Use this code at your own risk! It is provided without any guarantees of support, accuracy, or even that it will work! While I will do what I can to maintain things and address questions that arise, it is ultimately up to each user to build and maintain their copy of the code. And it is up to you, the user, to understand what the code is doing, and to be responsible for the content of your simulations. No numbers in e.g. the parameterfile and Config file examples are meant to be taken as recommendations: you should do considerable experimentation to determine the most appropriate values for the problem at hand. They will change depending on the problem details and what you are trying to extract.

Best of luck, enjoy!