LPAQA: Language model Prompt And Query Archive

This repository contains data and code for the paper How Can We Know What Language Models Know?

Install

Our repository is based on LAMA. Please download our fork from here and follow the instructions to set up the environment and download pre-trained language models.

# clone LPAQA
git clone https://github.com/jzbjyb/LPAQA.git LPAQA
# clone LAMA
git clone https://github.com/jzbjyb/LAMA.git LAMA
pushd LAMA && git reset --hard b6b1885c64de5981f249a8b65de25cb0802b4bd5 && rm -rf .git && popd
mv LAMA/* LPAQA/ && rm -rf LAMA
# follow the instructions to install
cd LPAQA
./setup.sh

Retrieve factual knowledge from LMs

For example, to query the owner of MSN (Microsoft is the answer), you can either use manually created prompts (x is owned by y):

python lama/eval_generation.py --lm bert --t "MSN is owned by [MASK]."

or use LPAQA that ensembles a diversity of prompts:

# mined prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/mine
# paraphrased prompts
python lama/eval_ensemble.py --lm bert --subject MSN --relation P127 --prompts prompt/paraphrase

Language model Prompt And Query Archive

Related tags

Overview

LPAQA: Language model Prompt And Query Archive

Install

Retrieve factual knowledge from LMs

Owner

A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility

Open-Ended Commonsense Reasoning (NAACL 2021)

4D Human Body Capture from Egocentric Video via 3D Scene Grounding

SCU OlympicsRunning Baseline

Anchor-free Oriented Proposal Generator for Object Detection

Codes for TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization.

An pytorch implementation of Masked Autoencoders Are Scalable Vision Learners

A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

An efficient toolkit for Face Stylization based on the paper "AgileGAN: Stylizing Portraits by Inversion-Consistent Transfer Learning"

[NeurIPS'20] Self-supervised Co-Training for Video Representation Learning. Tengda Han, Weidi Xie, Andrew Zisserman.

Action Segmentation Evaluation

PyTorch code for training MM-DistillNet for multimodal knowledge distillation

Interpretable and Generalizable Person Re-Identification with Query-Adaptive Convolution and Temporal Lifting

Progressive Domain Adaptation for Object Detection

EEGEyeNet is benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty

SPTAG: A library for fast approximate nearest neighbor search

Segment axon and myelin from microscopy data using deep learning

My coursework for Machine Learning (2021 Spring) at National Taiwan University (NTU)

PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners for self-supervised ViT.

Biomarker identification for COVID-19 Severity in BALF cells Single-cell RNA-seq data