Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated the Cube force routine [WIP] #51

Merged
merged 75 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
9814bc2
Preliminary check in of Cube force extension to EXP and pyEXP
Oct 26, 2023
61b207e
Merge remote-tracking branch 'origin/TensorGCC13' into CubeForce
Oct 26, 2023
64ab287
Debugging and updated pyEXP wrappers; added PeriodicBC
Oct 26, 2023
fc7a41e
Change some variable names for clarity
Oct 26, 2023
653d342
Added comments and namespace qualification only
Oct 26, 2023
b4c7756
Comments and clean up
Oct 26, 2023
e14268c
Moved readNativeCoefs implementation to source file for better debugging
Oct 26, 2023
99a173f
More comments
Oct 26, 2023
99a0d92
Fix typo for class qualification in wrapper
Oct 26, 2023
cf29eea
Added some dimension sanity checks
Oct 27, 2023
5986bfc
Missing <cassert> header
Oct 27, 2023
2586172
Missing assignment of dfac constant
Oct 27, 2023
e5389ee
Changed the warp-reduce so that it will work with thrust::complex
Oct 29, 2023
f74dcb4
A trivial cube IC generator
Oct 29, 2023
0dbbcaa
Increased precision in phase-space output values
Oct 29, 2023
b859b60
Updated template signature for warpReduce update
Oct 29, 2023
a14ba0a
Added a preliminary cuda implmentation for cube; this compiles but is…
Oct 29, 2023
7c5bda5
Exclude the GPU reporting if Cuda is not enabled
Oct 29, 2023
f42e1c5
Some additional comments
Oct 29, 2023
47878a8
Fix error in Cuda shared memory allocation for thrust::complex type
Oct 29, 2023
cccc961
Merge branch 'CubeForce' of github.com:EXP-code/EXP into CubeForce
Oct 29, 2023
abf4cf4
Wrong index type in the cuda force kernel
Oct 29, 2023
ce388c4
Missing pot accumulation on cuda side; don't accumulate the constant …
Oct 29, 2023
df13cf9
Enable coefficient dumps for Cube force
Oct 30, 2023
cf37ff3
Fix up grid dimension storage allocation for CubeStruct
Oct 30, 2023
f426eac
Simplify and correct coordinate wrapping for the cube in BasisFactory
Oct 30, 2023
f060bfc
Missing entry for 'cube' geometry
Oct 30, 2023
b24ccc0
Fix typo in yaml parser for cube
Oct 30, 2023
a2f1437
Various minor fixes to the pyEXP routines for Cube
Oct 30, 2023
ff1e3f2
Added some docs for the Cube basis
Oct 30, 2023
43cf1b5
Added a wave perturbation for coefficient testing
Oct 31, 2023
49272ed
Added some additional comments only
Oct 31, 2023
350a8d5
Operator typo in cuda coef setting
Nov 1, 2023
c3ae553
Fixes for GPU coefficient factor; and debug output code turned off
Nov 1, 2023
ed727a9
Some additional optimization for the cuda kernels using recursion rat…
Nov 2, 2023
d8578ab
Corrections to recursion relation algorithm
Nov 2, 2023
24908a6
Preliminary implementation of muliple coorindate systems for Basis an…
Nov 2, 2023
d4a106f
Additional documentation of the 'coord' argument in 'FieldGenerator'
Nov 2, 2023
7c7ab55
Use alias for readability
Nov 2, 2023
576e224
Some additional comments for future reference
Nov 2, 2023
93834ed
Change the default logic so provided parameters are not appended
Nov 3, 2023
1af1989
Missing coord parameter from FieldGenerator::volumes()
Nov 3, 2023
47ed432
Let the Basis set the natural coordinate type. This will be the righ…
Nov 3, 2023
6966c04
Wrong coordinate enum for Cube, oops
Nov 3, 2023
f7114a1
Implement cuda coefficient reduction in a single dimension at a time …
Nov 5, 2023
31b3803
OutCoef must have component name. Throw an exception if the coponent…
Nov 5, 2023
9f868a0
Add 'planes' parameter to BasisFactory. It is not used but allows an…
Nov 5, 2023
22f379f
New GPU algorithm checks. Making the GPU ram frugal method the default
Nov 6, 2023
f09755e
Make 'method' string case insensitive. 3d triggered by 'all', 'full'…
Nov 6, 2023
8bee3f3
Remove shocktube tracing from PeriodicBC; available as a user module
Nov 6, 2023
f96156c
Update default BC strategy for cube: no wrapping, all done by Periodi…
Nov 6, 2023
edfce7c
Fix to an erroneous signature for 'return_euler_slater' in Orient tha…
Nov 8, 2023
8129fe1
Merge branch 'CubeForce' of github.com:EXP-code/EXP into CubeForce
Nov 8, 2023
d14e8a3
Version bump
Nov 9, 2023
d43a270
Merge branch 'CubeForce' of github.com:EXP-code/EXP into CubeForce
Nov 9, 2023
534118b
Fixed cuda_initialize logic
Nov 15, 2023
1986298
Initial skeleton for CTests
Nov 15, 2023
68fcae3
Some additional comments, before I forget
Nov 15, 2023
4b438f2
Some additional tests
Nov 15, 2023
ee1532e
Improve test names
Nov 15, 2023
a2c7372
Added a few more example tests and added more comments
Nov 16, 2023
2e37634
Added a readme to the test directory
Nov 16, 2023
6b666a9
Added a few non-trivial pyEXP tests
Nov 16, 2023
af0f15e
Update README.md
The9Cat Nov 16, 2023
50ee0a1
Add log output for byPlanes toggle
The9Cat Nov 16, 2023
25cd338
Merge branch 'CubeForce' of github.com:EXP-code/EXP into CubeForce
The9Cat Nov 16, 2023
fd4ab63
- Updated CTest config for backward compatibility with earlier versio…
The9Cat Nov 16, 2023
1bd45c9
Added specific 'main' to on: block
Nov 16, 2023
9573388
Merge branch 'CTests' of github.com:EXP-code/EXP into CTests
Nov 16, 2023
25599d7
Merge branch 'CTests' into CubeForce
Nov 16, 2023
b7d1bea
Merge branch 'CubeForce' of github.com:EXP-code/EXP into CubeForce
Nov 16, 2023
b5cf1fb
Merge branch 'CTests' into CubeForce
Nov 16, 2023
6755cd3
Added zero mean velocity option
Nov 17, 2023
2f58132
Added Cube test
Nov 17, 2023
e6b6248
Check a file in the home directory rather than the test directory
Nov 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ on:
branches:
- main
pull_request:

branches:
- main
jobs:
pyexp:
strategy:
Expand Down
19 changes: 14 additions & 5 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
cmake_minimum_required(VERSION 3.15) # This is the oldest version I've
# tested but newer versions are
# preferred esp. for Cuda support
cmake_minimum_required(VERSION 3.21) # Needed for CUA, MPI, and CTest features

project(
EXP
VERSION "7.7.26"
VERSION "7.7.27"
HOMEPAGE_URL https://github.com/EXP-code/EXP
LANGUAGES C CXX Fortran)

Expand Down Expand Up @@ -41,16 +40,20 @@ option(ENABLE_NBODY "Enable EXP n-body" ON)
option(ENABLE_PYEXP "Enable the Python bindings" ON)
option(ENABLE_PNG "Enable PNG graphics support" FALSE)
option(ENABLE_CUDA "Enable CUDA" FALSE)
option(ENABLE_SLURM "Enable SLURM checkpointing support" TRUE)
option(ENABLE_SLURM "Enable SLURM checkpointing support" FALSE)
option(ENABLE_XDR "Enable RPC/XDR support for Tipsy standard" FALSE)
option(ENABLE_VTK "Configure VTK if available" FALSE)
option(ENABLE_CUDA_SINGLE "Use real*4 instead of real*8 for CUDA" FALSE)
option(ENABLE_DSMC "Enable DSCM module" FALSE)
option(ENABLE_USER "Enable basic user modules" ON)
option(ENABLE_SLCHECK "Enable *careful* Sturm-Liouville solutions" TRUE)
option(ENABLE_TESTS "Enable build tests for EXP, pyEXP and helpers" ON)
option(BUILD_SHARED_LIBS "Build using shared libraries" ON)
option(BUILD_DOCS "Build documentation" OFF)

# Set mpirun launcher for CTest

set(EXP_MPI_LAUNCH "mpirun" CACHE STRING "Command to run an MPI application (for unit tests only)")

# Find newest version if multiple versions are available

Expand Down Expand Up @@ -244,6 +247,12 @@ endif()

add_subdirectory(extern/user-modules)

# Build the tests; set ENABLE_TEST=OFF to disable
if(ENABLE_TESTS)
include(CTest)
add_subdirectory(tests)
endif()

# try to find pybind11 and build wrapper python module
find_package(Python3 COMPONENTS Interpreter Development)
message(STATUS "python3 include dirs: ${Python3_INCLUDE_DIRS}")
Expand Down
234 changes: 220 additions & 14 deletions coefs/BasisFactory.H
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

#include <ParticleReader.H>
#include <Coefficients.H>
#include <BiorthCube.H>
#include <SLGridMP2.H>
#include <YamlCheck.H>
#include <BiorthCyl.H>
Expand Down Expand Up @@ -87,11 +88,24 @@ namespace BasisClasses
double totalMass;

//! Evaluate fields in spherical coordinates in centered coordinate system
virtual void all_eval(double r, double costh, double phi,
virtual void all_eval_sph(double r, double costh, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& pott, double& potp) = 0;


//! Evaluate fields in cylindrical coordinates in centered coordinate system
virtual void all_eval_cyl(double r, double z, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& potz, double& potp) = 0;


//! Evaluate fields in Cartesian coordinates in centered coordinate system
virtual void all_eval_crt(double x, double y, double z,
double& den0, double& den1,
double& pot0, double& pot1,
double& potx, double& poty, double& potz) = 0;

//! Name of the derived class
std::string name;

Expand All @@ -107,6 +121,40 @@ namespace BasisClasses
//! Subspace index
virtual const std::string harmonic() = 0;

//! Desired coordinate system for evaluation (default: Spherical)
enum class Coord { Spherical, Cylindrical, Cartesian, None };

//! Set the correponding coorindate labels
static std::map<Coord, std::string> coordLabels;

//! Get field type
Coord parseFieldType(std::string Coord_type);

//! Current field coordindate type
Coord coordinates;

//! Get field labels
std::vector<std::string> getFieldLabels(const Coord ctype);

//! Evaluate basis in spherical coordinates
virtual void all_eval(double x1, double x2, double x3,
double& den0, double& den1,
double& pot0, double& pot1,
double& p1, double& p2, double& p3,
const Coord ctype=Coord::Spherical)
{
if (ctype==Coord::Spherical)
all_eval_sph(x1, x2, x3, den0, den1, pot0, pot1, p1, p2, p3);
else if (ctype==Coord::Cylindrical)
all_eval_cyl(x1, x2, x3, den0, den1, pot0, pot1, p1, p2, p3);
else if (ctype==Coord::Cartesian)
all_eval_crt(x1, x2, x3, den0, den1, pot0, pot1, p1, p2, p3);
else {
all_eval_sph(x1, x2, x3, den0, den1, pot0, pot1, p1, p2, p3);
p1 = p2 = p3 = 0.0;
};
};

public:

//! Constructor from YAML node
Expand Down Expand Up @@ -168,6 +216,13 @@ namespace BasisClasses
//! Provide a set of coefficients using a CoefStruct
virtual void set_coefs(CoefClasses::CoefStrPtr coefs) = 0;

//! Set field coordindate system
void setFieldType(std::string coord_type)
{ coordinates = parseFieldType(coord_type); }

//! Get current field coordinate type
std::string getFieldType() { return coordLabels[coordinates]; }

//@{
//! The coefficient factory. Creates a shared pointer to derived
//! instance of the desired type and returns it.
Expand Down Expand Up @@ -231,12 +286,6 @@ namespace BasisClasses

protected:

//! Evaluate basis in spherical coordinates
virtual void all_eval(double r, double costh, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& pott, double& potp);

//! Load coefficients into the new CoefStruct
virtual void load_coefs(CoefClasses::CoefStrPtr coefs, double time);

Expand All @@ -252,6 +301,25 @@ namespace BasisClasses
//! Subspace index
virtual const std::string harmonic() { return "l";}

//! Evaluate basis in spherical coordinates
virtual void all_eval_sph(double r, double costh, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& pott, double& potp);

//! Evaluate basis in cartesian coordinates
virtual void all_eval_crt(double x, double y, double z,
double& den0, double& den1,
double& pot0, double& pot1,
double& potx, double& poty, double& potz);


//! Evaluate basis in cylindrical coordinates
virtual void all_eval_cyl(double R, double z, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potR, double& potz, double& potp);

public:

//! Constructor from YAML node
Expand Down Expand Up @@ -356,19 +424,25 @@ namespace BasisClasses
protected:

//! Evaluate basis in cylindrical coordinates
void all_eval_cyl
virtual void all_eval_cyl
(double R, double z, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potR, double& potz, double& potp);

//! Evaluate basis in spherical coordinates. Conversion from the
//! cylindrical evaluation above.
virtual void all_eval(double r, double costh, double phi,
virtual void all_eval_sph(double r, double costh, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& pott, double& potp);

// Cartesian
virtual void all_eval_crt(double x, double y, double z,
double& den0, double& den1,
double& pot0, double& pot1,
double& potx, double& poty, double& potz);

//! Load coefficients into the new CoefStruct
virtual void load_coefs(CoefClasses::CoefStrPtr coefs, double time);

Expand Down Expand Up @@ -500,10 +574,22 @@ namespace BasisClasses
protected:

//! Evaluate basis in spherical coordinates
virtual void all_eval(double r, double costh, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& pott, double& potp);
virtual void all_eval_sph(double r, double costh, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& pott, double& potp);

//! Evaluate basis in cylindrical coordinates
virtual void all_eval_cyl(double r, double z, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& potz, double& potp);

//! Evaluate basis in cartesian coordinates
virtual void all_eval_crt(double x, double y, double z,
double& den0, double& den1,
double& pot0, double& pot1,
double& potx, double& poty, double& potz);

//! Load coefficients into the new CoefStruct
virtual void load_coefs(CoefClasses::CoefStrPtr coefs, double time);
Expand Down Expand Up @@ -577,6 +663,126 @@ namespace BasisClasses
}
};

/**
Uses the BiorthCyl basis to evaluate expansion coeffients and
provide potential and density basis fields
*/
class Cube : public Basis
{

public:

using BasisMap = std::map<std::string, Eigen::VectorXd>;
using BasisArray = std::vector<std::vector<BasisMap>>;

private:

//! Wave function constant
static constexpr std::complex<double> kfac{0.0, 2.0*M_PI};

//! Initialization helper
void initialize();

//! Orthogonal basis instance
std::shared_ptr<BiorthCube> ortho;

//! Minimum expansion order for restriction
int nminx, nminy, nminz;

//! Maximum expansion order for construction
int nmaxx, nmaxy, nmaxz;

//! Number of integration knots for orthogonal check
int knots;

//! Number of particles used to compute grid
unsigned used;

using coefType = Eigen::Tensor<std::complex<double>, 3>;

coefType expcoef;

//! Notal mass on grid
double totalMass;

//! Number of particles
int npart;

protected:

//! Evaluate basis in Cartesian coordinates
virtual void all_eval_crt
(double x, double y, double z,
double& den0, double& den1,
double& pot0, double& pot1,
double& potx, double& poty, double& potz);

//! Evaluate basis in spherical coordinates. Conversion from the
//! Cartesian evaluation above.
virtual void all_eval_sph(double r, double costh, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& pott, double& potp);

//! Evaluate basis in cylindrical coordinates
virtual void all_eval_cyl(double r, double z, double phi,
double& den0, double& den1,
double& pot0, double& pot1,
double& potr, double& potz, double& potp);


//! Load coefficients into the new CoefStruct
virtual void load_coefs(CoefClasses::CoefStrPtr coefs, double time);

//! Set coefficients
virtual void set_coefs(CoefClasses::CoefStrPtr coefs);

//! Valid keys for YAML configurations
static const std::set<std::string> valid_keys;

//! Return readable class name
virtual const std::string classname() { return "Cube";}

//! Readable index name
virtual const std::string harmonic() { return "n";}

public:

//! Constructor from YAML node
Cube(const YAML::Node& conf);

//! Constructor from YAML string
Cube(const std::string& confstr);

//! Destructor
virtual ~Cube(void) {}

//! Zero out coefficients to prepare for a new expansion
void reset_coefs(void);

//! Make coefficients after accumulation
void make_coefs(void);

//! Accumulate new coefficients
virtual void accumulate(double x, double y, double z, double mass);

//! Get potential, density and forces at a point
virtual void getFields
(double x, double y, double z,
double& tdens0, double& tpotl0, double& tdens, double& tpotl,
double& tpotx, double& tpoty, double& tpotz);


//! Return current maximum harmonic order in expansion
Eigen::Vector3i getNmax() { return {nmaxx, nmaxy, nmaxz}; }

//! Compute the orthogonality of the basis by returning inner
//! produce matrices
Eigen::MatrixXcd orthoCheck();

};


//! Time-dependent potential-density model
using BasisCoef = std::tuple<std::shared_ptr<Basis>, std::shared_ptr<CoefClasses::Coefs>>;

Expand Down
Loading
Loading