Issue with running DFT job from Video Tutorials (4/5)

[kimiaGF]

Hi everyone,

I am using the binder to follow along with the video tutorials and have run into an issue with running DFT jobs in the notebook. In the video, the following is typed in:

for strain in strain_list:
    job_strain = job_dft.copy_to(new_job_name=f'spx_{1+strain}'.replace('.','_'),
                                 new_database_entry = False
                                )
    structure_copy = structure.copy()
    structure_copy.apply_strain(strain)
    job_strain.structure = structure_copy
    job_strain.run()

However I get the following Value error:

ValueError                                Traceback (most recent call last)
Cell In[47], line 8
      6 structure_copy.apply_strain(strain)
      7 job_strain.structure = structure_copy
----> 8 job_strain.run()

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/utils/deprecate.py:171, in Deprecator.__deprecate_argument.<locals>.decorated(*args, **kwargs)
    161     if kw in self.arguments:
    162         warnings.warn(
    163             message_format.format(
    164                 "{}.{}({}={})".format(
   (...)
    169             stacklevel=2,
    170         )
--> 171 return function(*args, **kwargs)

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/jobs/job/generic.py:704, in GenericJob.run(self, delete_existing_job, repair, debug, run_mode, run_again)
    702     self._run_if_repair()
    703 elif status == "initialized":
--> 704     self._run_if_new(debug=debug)
    705 elif status == "created":
    706     self._run_if_created()

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/jobs/job/generic.py:1238, in GenericJob._run_if_new(self, debug)
   1230 def _run_if_new(self, debug=False):
   1231     """
   1232     Internal helper function the run if new function is called when the job status is 'initialized'. It prepares
   1233     the hdf5 file and the corresponding directory structure.
   (...)
   1236         debug (bool): Debug Mode
   1237     """
-> 1238     run_job_with_status_initialized(job=self, debug=debug)

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/jobs/job/runfunction.py:69, in run_job_with_status_initialized(job, debug)
     60 def run_job_with_status_initialized(job, debug=False):
     61     """
     62     Internal helper function the run if new function is called when the job status is 'initialized'. It prepares
     63     the hdf5 file and the corresponding directory structure.
   (...)
     67         debug (bool): Debug Mode
     68     """
---> 69     job.validate_ready_to_run()
     70     if job.server.run_mode.queue:
     71         job.check_setup()

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_atomistics/sphinx/base.py:1571, in SphinxBase.validate_ready_to_run(self)
   1564 def validate_ready_to_run(self):
   1565     """
   1566     Checks whether parameters are set appropriately. It does not
   1567     mean the simulation won't run if it returns False.
   1568     """
   1570     all_groups = {
-> 1571         "job.input.pawPot": self.input.sphinx.pawPot,
   1572         "job.input.structure": self.input.sphinx.structure,
   1573         "job.input.basis": self.input.sphinx.basis,
   1574         "job.input.PAWHamiltonian": self.input.sphinx.PAWHamiltonian,
   1575         "job.input.initialGuess": self.input.sphinx.initialGuess,
   1576         "job.input.main": self.input.sphinx.main,
   1577     }
   1579     if np.any([len(all_groups[group]) == 0 for group in all_groups]):
   1580         self.load_default_groups()

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/storage/datacontainer.py:387, in DataContainer.__getattr__(self, name)
    382 def __getattr__(self, name):
    383     # this is only called when python doesn't find name in the instance
    384     # or class variables, so we don't need to go through the same lengths
    385     # here as in __setattr__
    386     try:
--> 387         return self[name]
    388     except KeyError:
    389         raise AttributeError(name) from None

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/storage/datacontainer.py:315, in DataContainer.__getitem__(self, key)
    313         return v
    314     else:
--> 315         v = self._store[self._indices[key]] = v.load()
    316         return v
    317 except KeyError:

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/storage/hdfstub.py:96, in HDFStub.load(self)
     86 def load(self):
     87     """
     88     Read value from HDF.
     89 
   (...)
     92     Otherwise it will be loaded with :method:`.ProjectHDFio.to_object()`.
     93     """
     94     if (
     95         self._group_name in self._hdf.list_nodes()
---> 96         or "TYPE" not in self._hdf[self._group_name].list_nodes()
     97     ):
     98         return self._hdf[self._group_name]
    100     load = self._load_functions.get(
    101         self._hdf[self._group_name]["TYPE"], lambda h, g: h[g].to_object()
    102     )

File /srv/conda/envs/notebook/lib/python3.10/site-packages/pyiron_base/storage/hdfio.py:172, in FileHDFio.__getitem__(self, item)
    170                 obj = self._convert_dtype_obj_array(obj)
    171             return obj
--> 172     raise ValueError(
    173         "Unknown item: {} {} {}".format(item, self.file_name, self.h5_path)
    174     )
    175 else:
    176     if (
    177         item_lst[0] == ""
    178     ):  # item starting with '/', thus we have an absoute HDF5 path

ValueError: Unknown item: sphinx__index_0 /home/jovyan/dft/spx_0_95.h5 /spx_0_95/input/parameters

I have combed through everything and compared it to the video and it is identical.

Kimia Ghaffari

[jan-janssen]

@kimiaGF I am sorry for the delay, it seems like there is a bug in the copy_to() function. Can you try the following?

for strain in strain_lst: 
    job_strain = pr.create.job.Sphinx(job_name="spx_" + str(1+strain).replace(".", "_"))
    job_strain.structure = structure
    # job_strain = job_dft.copy_to(
    #     new_job_name="spx_" + str(1+strain).replace(".", "_"), 
    #     new_database_entry=False
    # )
    structure_copy = structure.copy()
    structure_copy.apply_strain(strain)
    job_strain.structure =structure_copy 
    job_strain.run()

Also you might need to delete the existing calculation as the failed calculation might confuse pyiron. You can do this with pr.remove_jobs(recursive=True, silently=True). @samwaseda Can you take a look why the copy_to() no longer works with SphinxDFT? It seems to be related to the lazy loading.

[samwaseda]

Sorry I'm on vacation right now and don't react very quickly, but I'll try to fix it as quickly as possible.

[samwaseda]

I think @jan-janssen's code should also work, but maybe this is even shorter and more elegant:

for strain in strain_lst: 
    job_strain = pr.create.job.Sphinx(job_name=("spx", strain))
    job_strain.structure = structure.apply_strain(strain, return_box=True)
    job_strain.run()

[samwaseda]

I realized that I could find a remedy for this problem, but it won't remove the cause of the problem, which is discussed here. So if it's ok, I would like to ask you use the code I posted above for now. I'll try to find a solution as quickly as possible.

[kimiaGF]

Thank you @jan-janssen and @samwaseda! Both versions of the above code work for me now. I appreciate the help 😄