Preprocessing script for HighD dataset

[achintyanath]

I'm impressed by your work. Could you please share the preprocessing script for the HighD dataset? I'd like to reproduce your results.

[IndefiniteBen]

There is a script in the older repo, but it isn't documented how to reproduce their processing.

See this issue: Petrichor625/BATraj-Behavior-aware-Model#5

[Petrichor625]

Apologies for the delay, I've been quite busy lately and couldn't respond promptly.
HighD dataset requires online application, and as per the official agreement, we can't directly provide it. You'll need to apply for it yourself, and processing will align with stdan, as detailed in HLTP's Issue and Readme sections. I've provided instructions there.

[IndefiniteBen]

Yes, I have applied and received the HighD data. We are not asking for you to provide the HighD data.

We are only asking about processing it from the original dataset format (created by the authors of HighD) into the format needed for this repo.

Can you please be more specific with this?

processing will align with stdan, as detailed in HLTP's Issue and Readme sections. I've provided instructions there.

If I look at the stdan repo I cannot find any instructions for processing the data. The only thing the HLTP readme says is that the format is the same as stdan. Here is an open issue on stdan where they are also asking for the processing steps. Can you please provide specific links or quotes, because I have looked but am yet to find and instructions.

[zzt007]

@IndefiniteBen
Hi,
I have the same need as you ，have you found the highD process file yet?

[IndefiniteBen]

@zzt007 Negative.

@Petrichor625 does not seem to check GitHub very often, so we will likely have to wait.

If you figure it out please share!

[zzt007]

@IndefiniteBen
Alright. I will try to understand how the highD_process.m file work .

[IndefiniteBen]

Did you already see this other issue I posted. Perhaps there is some useful information there, for your search?

[zzt007]

@IndefiniteBen
yep，thanks for your posted.
But I found another file to process the HighD dataset , I thought it might be useful for u , here is the link :
https://github.com/nqyy/lane-change-prediction-lstm/blob/master/process_data.py

[Petrichor625]

Sorry for the delayed response. Our project is still ongoing, and for now, we’re not planning to open-source the preprocessing code for the highD data. Thank you for your understanding!

If you encounter any other issues while running HLTP, please feel free to reach out. Since I may not always receive GitHub notifications or be able to check messages promptly, if my response is delayed, please don’t hesitate to contact me via email. We’ll do our best to assist you!

[Petrichor625]

Thank you so much for your patience and interest in our project!

We’re happy to let you know that we’ve just uploaded a fully updated version of the code, along with all necessary model parameters and weight files. This new version should address some previous issues and allow you to replicate the results of NGSIM datasets.

We genuinely appreciate your support and understanding as we worked to make these updates available. If you have any further questions or run into any other issues, please feel free to reach out—we’re here to help!

Thank you once again for your patience, and we hope the updated code is helpful for your work!

[jeffei]

Yes, I have applied and received the HighD data. We are not asking for you to provide the HighD data.

We are only asking about processing it from the original dataset format (created by the authors of HighD) into the format needed for this repo.

Can you please be more specific with this?

processing will align with stdan, as detailed in HLTP's Issue and Readme sections. I've provided instructions there.

If I look at the stdan repo I cannot find any instructions for processing the data. The only thing the HLTP readme says is that the format is the same as stdan. Here is an open issue on stdan where they are also asking for the processing steps. Can you please provide specific links or quotes, because I have looked but am yet to find and instructions.

@IndefiniteBen
It is a big pity, I have followed several papers that announced their achieved sota in Ngsim and highD datasets, but none of them shared the preprocessing script even preprocessed data.

If you have any clue about highD dataset preprocessing details, please share them with me. Thanks in advance!