To use the benchmark templates, simply access and use the benchmark folder.

To reproduce some of the results in the paper or to use already obtained results from CoreNLP, Spacy, and Flair models on benchmarks, simply run the code.py by giving the results-set you are interested in and the name of the output file as input arguments.

Example: python code.py ./results/Template_1/results_flair out Will produce results from Flair model on Template number 1 and will output the results in the out file.

Referenced paper link: https://arxiv.org/abs/1910.10872

cite
@article{mehrabi2019man,
  title={Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition},
  author={Mehrabi, Ninareh and Gowda, Thamme and Morstatter, Fred and Peng, Nanyun and Galstyan, Aram},
  journal={arXiv preprint arXiv:1910.10872},
  year={2019}
}