Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ml-run-process ms4alg.sort, Warning: unable to rename file #57

Open
bl202 opened this issue Jul 30, 2018 · 6 comments
Open

ml-run-process ms4alg.sort, Warning: unable to rename file #57

bl202 opened this issue Jul 30, 2018 · 6 comments

Comments

@bl202
Copy link

bl202 commented Jul 30, 2018

Hi, I just installed mountainlab-js through conda and am trying out your algorithm but I keep getting the following error

RUNNING: ml-run-process ephys.bandpass_filter --inputs timeseries:/home/bl202/Documents/sample_data/raw.mda --parameters freq_max:6000 freq_min:300 samplerate:30000 --outputs timeseries_out:/home/bl202/Documents/sample_data/out/filt.mda.prv
[ Getting processor spec... ]
[ Checking inputs and substituting prvs ... ]
[ Computing process signature ... ]
[ Checking outputs... ]
[ Checking process cache ... ]
[ Process ephys.bandpass_filter already completed. ]
[ Creating output prv for timeseries_out ... ]
[ Done. ]
RUNNING: ml-run-process ephys.whiten --inputs timeseries:/home/bl202/Documents/sample_data/out/filt.mda.prv --parameters --outputs timeseries_out:/home/bl202/Documents/sample_data/out/pre.mda.prv
[ Getting processor spec... ]
[ Checking inputs and substituting prvs ... ]
Locating /home/bl202/Documents/sample_data/out/filt.mda.prv ...
[ Computing process signature ... ]
[ Checking outputs... ]
[ Checking process cache ... ]
[ Process ephys.whiten already completed. ]
[ Creating output prv for timeseries_out ... ]
[ Done. ]
RUNNING: ml-run-process ms4alg.sort --inputs geom:/home/bl202/Documents/sample_data/geom.csv timeseries:/home/bl202/Documents/sample_data/out/pre.mda.prv --parameters adjacency_radius:100 detect_sign:-1 detect_threshold:-1 --outputs firings_out:output/firings.mda
[ Getting processor spec... ]
Warning: unable to rename file /home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC/output_firings_out.mda -> /home/bl202/Documents/output/firings.mda . Perhaps temporary directory is not on the same device as the output file directory.
[ Checking inputs and substituting prvs ... ]
Error renaming file /home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC/output_firings_out.mda -> /home/bl202/Documents/output/firings.mda: ENOENT: no such file or directory, copyfile '/home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC/output_firings_out.mda' -> '/home/bl202/Documents/output/firings.mda'
Locating /home/bl202/Documents/sample_data/out/pre.mda.prv ...
[ Computing process signature ... ]
[ Checking outputs... ]
[ Checking process cache ... ]
[ Creating temporary directory ... ]
[ Preparing temporary outputs... ]
[ Initializing process ... ]
[ Running ... ] /home/bl202/anaconda3/envs/mlab/bin/python3 /home/bl202/anaconda3/envs/mlab/etc/mountainlab/packages/ml_ms4alg/ms4alg_spec.py.mp ms4alg.sort --geom=/home/bl202/Documents/sample_data/geom.csv --timeseries=/home/bl202/Documents/tmp-mlab/output_fc01c6308d08b94c019ddc06dc9ff6e6debf8494_timeseries_out.mda --firings_out=/home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC/output_firings_out.mda --adjacency_radius=100 --detect_sign=-1 --detect_threshold=-1 --_tempdir=/home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC
Using tempdir=/home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC
Preparing /home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC/timeseries.hdf5...
Preparing neighborhood sorters...

Detecting events on channel 1 (phase1)...
Detecting events on channel 2 (phase1)...

Detecting events on channel 4 (phase1)...
Detecting events on channel 3 (phase1)...

Computing PCA features for channel 3 (phase1)...

Computing PCA features for channel 4 (phase1)...

Computing PCA features for channel 1 (phase1)...

Computing PCA features for channel 2 (phase1)...

Clustering for channel 3 (phase1)...

Clustering for channel 2 (phase1)...

Clustering for channel 1 (phase1)...

Clustering for channel 4 (phase1)...

Found 3 clusters for channel 1 (phase1)...
Computing templates for channel 1 (phase1)...

Found 2 clusters for channel 4 (phase1)...
Computing templates for channel 4 (phase1)...

Found 2 clusters for channel 3 (phase1)...
Computing templates for channel 3 (phase1)...

Found 2 clusters for channel 2 (phase1)...
Computing templates for channel 2 (phase1)...

Re-assigning events for channel 1 (phase1)...

Re-assigning events for channel 4 (phase1)...

Re-assigning events for channel 3 (phase1)...

Re-assigning events for channel 2 (phase1)...

Computing PCA features for channel 3 (phase2)...

Computing PCA features for channel 1 (phase2)...

Computing PCA features for channel 4 (phase2)...

Computing PCA features for channel 2 (phase2)...

Clustering for channel 3 (phase2)...

Clustering for channel 1 (phase2)...

Clustering for channel 4 (phase2)...

Clustering for channel 2 (phase2)...

Found 4 clusters for channel 3 (phase2)...

Found 6 clusters for channel 1 (phase2)...

Found 12 clusters for channel 4 (phase2)...

Found 9 clusters for channel 2 (phase2)...

Preparing output...

Writing firings file...

Done.

Elapsed time for processor ms4alg.sort: 99.194 sec
Finalizing output firings_out
[ Removing temporary directory ... ]
Traceback (most recent call last):

**File "", line 1, in
runfile('/home/bl202/Documents/temp.py', wdir='/home/bl202/Documents')

File "/home/bl202/anaconda3/envs/mlab/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 678, in runfile
execfile(filename, namespace)

File "/home/bl202/anaconda3/envs/mlab/lib/python3.6/site-packages/spyder_kernels/customize/spydercustomize.py", line 106, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "/home/bl202/Documents/temp.py", line 259, in
detect_threshold=-1,

File "/home/bl202/Documents/temp.py", line 54, in sort_dataset
opts=opts

File "/home/bl202/Documents/temp.py", line 125, in ms4alg_sort
opts

File "/home/bl202/anaconda3/envs/mlab/lib/python3.6/site-packages/mountainlab_pytools/mlproc/mlproc_impl.py", line 235, in runProcess
return P.run(inputs,outputs,parameters,opts)

File "/home/bl202/anaconda3/envs/mlab/lib/python3.6/site-packages/mountainlab_pytools/mlproc/mlproc_impl.py", line 189, in run
raise Exception('Non-zero exit code for {}'.format(self.name()))

Exception: Non-zero exit code for ms4alg.sort**

As you can see, the temporary directory is in fact in the same device as the output file directory. Do you have any pointers on what I'm doing wrong?

Thanks in advance
@tjd2002
Copy link
Collaborator

tjd2002 commented Jul 30, 2018 via email

@tjd2002
Copy link
Collaborator

tjd2002 commented Jul 30, 2018

The previous issue is #33

@bl202
Copy link
Author

bl202 commented Jul 31, 2018

Thanks for the reply. Was using Mountainsort 4.1.0, just installed 4.2.0 and I get the same error. I'm attaching output of conda list.

conda_packs.log
console_ms4.2.log

@magland
Copy link
Collaborator

magland commented Jul 31, 2018

Could you please try this with a different temporary directory? Try using one that is on the same device as your output files. (see ml-config)

@bl202
Copy link
Author

bl202 commented Jul 31, 2018

Hi, I have tried other output directories (through defining the environment variable in mountainlab.env), and currently both the output and the tmp directories are in the same device:

/home/bl202/Documents/tmp-mlab/tempdir_d5c04fe186_mR0sjC/output_firings_out.mda
/home/bl202/Documents/output/firings.mda

@tjd2002
Copy link
Collaborator

tjd2002 commented Aug 1, 2018 via email •
edited
Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tjd2002 @magland @bl202