Failure to create dataset after editing "path" in exported dataset.json file

[namitha-sh]

Hello!

I was trying to create an example dataset along with the dataset.json file and the annotation_project.json file as shown in the example data. I was able to upload the audio directory into Whombat and then download the json file with all the metadata to create a dataset.json file. However the "path" for each audio file in the json file is the full path including the path to the audio directory:

In order for a second person to import this dataset, I manually edited the paths in the dataset.json file to include just the names of the audio files, because when they import the dataset, the path to the audio_dir in their local machine is specified differently:

However this failed to create the dataset even after specifying the correct path to the audio directory when uploading to Whombat.
Would you be able to provide some insight on this issue?

[mbsantiago]

Hi @namitha-sh!

Many thanks for reporting the issue. Ideally, as you mention, full paths should not be exported in order to facilitate imports on other machines. I'll have a look into why this is happening an report back soon.

[namitha-sh]

Hi @mbsantiago

Thank you so much for looking into the issue. I tried out the version after the merge was made and it seems the path is still exported relative to the home directory, not the audio directory itself. I guess if all the audio files are in the home directory, this works but not if they are within another audio directory like in example_data

[mbsantiago]

Hey @namitha-sh! Quick question, how are you running whombat? Did you download from PyPI (i.e. via pip), did you use one of the bundled versions in the releases section, or did you run directly from the source code? I haven't yet created a new release after the merge, so the PyPI and bundled versions are not updated.

[namitha-sh]

Hi @mbsantiago
I ran it directly from the source code, not the bundle or the PyPI. The problem also persists in the annotation project json file as well.

[mbsantiago]

ok thanks for the clarification. Will look into it.

[mbsantiago]

Hi @namitha-sh, I think I found the issue. I've implemented a proposed solution. Let me know if it works for you.

[namitha-sh]

Hi @mbsantiago

Thank you so much for making the changes, the paths in the dataset.json is working now and is only showing the path relative to the audio_dir, which is great. But the problem persists in other json files downloaded -- like the annotation_project.json and the validation.json files.

[mbsantiago]

Thanks @namitha-sh. If the dataset export is working the others should be an easy fix. Will implement those soon.

[namitha-sh]

Hi @mbsantiago ! Just wondering if you had any update on this. Also, while the paths are being exported correctly to the dataset.json file now, it still has problems when a new machine imports that file to create the dataset from the associated audio folder. It just says "Failed to create dataset". So perhaps Whombat is still not recognising the path?