It's a great tools, but when I try to transfer my .bib file, which is generated by an application BibDesk, the references are filtered, here is a minimal example of my bib file.

@inproceedings{zhang2019heterogeneous,
        author = {Zhang, Chuxu and Song, Dongjin and Huang, Chao and Swami, Ananthram and Chawla, Nitesh V},
        booktitle = {Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining},
        date-added = {2021-04-03 01:39:20 +0800},
        date-modified = {2021-04-03 01:44:13 +0800},
        keywords = {Recommender system, Graph Neural Network},
        pages = {793--803},
        title = {Heterogeneous graph neural network},
        year = {2019},
        Bdsk-Url-1 = {https://doi.org/10.1145/3292500.3330961}}

I think this is due to load_bib_file. The last line of this reference contains {, so load_bib_file skipped this reference.

However, in the BibtexParser, this kind of bib file can be recognized.

Hi,

Thanks for the great tool! I faced an issue where an entry was deleted after using rebiber though. It's this one:

@article{loon,
  title={Autonomous navigation of stratospheric balloons using reinforcement learning.},
  author={Marc G. Bellemare and Salvatore Candido and P. S. Castro and J. Gong and Marlos C. Machado and Subhodeep Moitra and Sameera S. Ponda and Ziyu Wang},
  journal={Nature},
  year={2020},
  volume={588 7836},
  pages={
          77-82
        }
}

Do you have any idea what could be wrong?

I come across two issues here:

• [ ] Somehow the tool transforms my comments (ones starting with %) in bib file into @comment{} and places them at the head of the file;
• [ ] All the bibs are by default organized in an alphabetic manner, is there a way (option) I can remain the order of bibs (and thus keep the comments where they are) .

Great tool by the way :)

When a bib_entry is not found now the scripts checks if it is an arxiv entry. If that is the case the new script adds the field url = {https://arxiv.org/abs/<ID>} to the bibitex entry.

Nice tool! It seems that currently it doesn't handle @string in BibTeX. Any plan to add this feature?

Example:

@string{emnlp = "Empirical Methods in Natural Language Processing (EMNLP)"}

@inproceedings{li2020efficient,
 title={Efficient One-Pass End-to-End Entity Linking for Questions},
 author={Li, Belinda Z. and Min, Sewon and Iyer, Srinivasan and Mehdad, Yashar and Yih, Wen-tau},
 booktitle=emnlp,
 year={2020}
}

Hello, I find that some papers accepted by some conferences (e.g. AAAI 2020) cannot be indexed. The reason might be that we can only download the first 1000 entries when the accepted papers are more than 1,000 from DBLP. Is there any way to address such problem? Thanks very much!

Although a scripting approach is provided, would you consider providing a Python API ？

for example

import rebiber
str = '@article{lin2020birds,
	title={Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models},
	author={Lin, Bill Yuchen and Lee, Seyeon and Khanna, Rahul and Ren, Xiang},
	journal={arXiv preprint arXiv:2005.00683},
	year={2020}
}'
res = rebiber.trans(str)
print(res)

This is notebook implemented a few modifications to the original Colab demo: (1) takes pasted BibTex as a string input (i.e. copying BibText from Google Scholar); (2) prints the processed BibTex on screen, which can be copied and pasted into reference management software; (3) modified the upload cell; (4) added a download cell to download processed outputs.

I've added data from NeurIPS 2020/2021, ICML 2021, ICLR 2021, and also missing entries from ICML 2020 and AAAI2019/2020, because they each have more than 1000 papers but currently each bib only contains the first 1000 entries.

The bug is as follows. When the input is something like:

@article{sharma19_paral_restar_spider, author = {Sharma, Pranay and Kafle, Swatantra and Khanduri, Prashant and Bulusu, Saikiran and Rajawat, Ketan and Varshney, Pramod K.}, title = {Parallel Restarted Spider -- Communication Efficient Distributed Nonconvex Optimization With Optimal Computation Complexity}, journal = {arXiv preprint arXiv:1912.06036}, year = 2019, url = {http://arxiv.org/abs/1912.06036v2}, archivePrefix = {arXiv}, primaryClass = {math.OC}, }

It is changed to:

@article{sharma19_paral_restar_spider, author = {Sharma, Pranay and Kafle, Swatantra and Khanduri, Prashan}, journal = {ArXiv preprint}, title = {Parallel Restarted Spider -- Communication Efficien}, url = {https://arxiv.org/abs/1912.06036}, volume = {abs/1912.06036}, year = {2019}

Author information is lost and the title is shortened to just the first line. This happens because if the entry is an unmatched article from arXiv, the normalizer re-parses the bibtex entry manually looking for the title, author, and arXiv id information. This manual parser fails when any of the entries (title/author/etc.) spans multiple lines. Moreover, the external bibtex parsing library used elsewhere already works quite well. This commit just uses the external parser which can handle multiline fields just fine. Under this, we have the correct output:

@article{sharma19_paral_restar_spider, author = {Sharma, Pranay and Kafle, Swatantra and Khanduri, Prashant and Bulusu, Saikiran and Rajawat, Ketan and Varshney, Pramod K.}, journal = {ArXiv preprint}, title = {Parallel Restarted Spider -- Communication Efficient Distributed Nonconvex Optimization With Optimal Computation Complexity}, url = {https://arxiv.org/abs/1912.06036}, volume = {abs/1912.06036}, year = {2019}

abbr.tsv has two entries for ICML:

Proc. of ICML | Proceedings of the .* International Conference on Machine Learning
Proc. of ICML | Machine Learning, Proceedings of the .* International Conference

but they are not both loaded, because in load_abbr_tsv() a dictionary is used such that the second entry overwrites the first one:

ls = line.split("|")
if len(ls) == 2:
    abbr_dict[ls[0].strip()] = ls[1].strip()

I see two solutions here: either don't load the file into a dictionary (but just a list of tuples), or allow specifying or regex patterns (i.e., (pattern1|pattern2)) which would require using a different character than | to separate the left- and right-hand sides in abbr.tsv.

Hi Yuchen, It seems you try to ignore the month field in a bib entry in is_contain_var() and build_json(). Can you please explain why is that necessary? You also ignore '@string' entry. Why not just let bibtexparser parse the entire bib file? Thank you!

Add entries for the following conferences (mostly machine learning conferences):

ICML 2022 AISTATS 2022 COLT 2021 2022 ICLR 2022 MLSYS 2021 2022 NeurIPS 2021 UAI 2021

I have multiple bib files for several research fields and hope convert their information in one-click. I've written a bat file to automatically execute bib files in work directory:

@echo off
for %%i in (*.bib) do echo "%%i"
for %%i in (*.bib) do rebiber -i %%i -o Pub%%i
pause
exit

But a build-in command would be easier to use. Would you like to add this?

I found that the booktitle of many papers in DBLP has too many names and information.

For example：

@inproceedings{seo-etal-2016-bidirectional,
 author = {Min Joon Seo and
Aniruddha Kembhavi and
Ali Farhadi and
Hannaneh Hajishirzi},
 bibsource = {dblp computer science bibliography, https://dblp.org},
 biburl = {https://dblp.org/rec/conf/iclr/SeoKFH17.bib},
 booktitle = {5th International Conference on Learning Representations, {ICLR} 2017,
Toulon, France, April 24-26, 2017, Conference Track Proceedings},
 publisher = {OpenReview.net},
 timestamp = {Thu, 25 Jul 2019 01:00:00 +0200},
 title = {Bidirectional Attention Flow for Machine Comprehension},
 url = {https://openreview.net/forum?id=HJ0UKP9ge},
 year = {2017}
}

The booktitle here contains the full name and abbreviation of ICLR, as well as their location. Can you keep only the first one of this information?

For example: booktitle =“5th International Conference on Learning Representations”

The ImageNet paper has its last author listed as Li Fei-Fei, which is how she publishes in general, both on the paper and in the IEEE metadata; their .bib has her as Li Fei-Fei in the author.

The DBLP record lists her as Li Fei{-}Fei.

And yet rebiber/data/cvpr2009.bib.json has her as Fei{-}Fei Li, and so running either through rebiber incorrectly changes it to that ordering.

The same is true for most (but not all) of her papers in the database. No idea why this would be, since DBLP consistently has her as Li Fei{-}Fei.

cc @pranav-ust

The LREC Sign Language workshop has this website - https://www.sign-lang.uni-hamburg.de/lrec/index.html

Which links to two bib files: without abstracts: https://www.sign-lang.uni-hamburg.de/lrec/sign-lang_lrec.bib with abstracts: https://www.sign-lang.uni-hamburg.de/lrec/sign-lang_lrec_a.bib

While one can add them manually to this repo, I was wondering if there is a setting somewhere to just put this link, and whenever someone runs an "update" script it will re-fetch the bib file and process it?