A PyTorch implementation of unsupervised SimCSE

Last update: Dec 23, 2022

Overview

A PyTorch implementation of unsupervised SimCSE

SimCSE: Simple Contrastive Learning of Sentence Embeddings

1. 用法

无监督训练

python train_unsup.py ./data/news_title.txt ./path/to/huggingface_pretrained_model

详细参数

python train_unsup.py -h

相似文本检索测试

python test_unsup.py

query title:
基金亏损路未尽 后市看法仍偏谨慎

sim title:
基金亏损路未尽 后市看法仍偏谨慎
海通证券：私募对后市看法偏谨慎
连塑基本面不容乐观 后市仍有下行空间
基金谨慎看待后市行情
稳健投资者继续保持观望 市场走势还未明朗
下半年基金投资谨慎乐观
华安基金许之彦：下半年谨慎乐观
楼市主导 期指后市不容乐观
基金公司谨慎看多明年市
前期乐观预期被否 基金重归谨慎

STS-B数据集训练和测试

中文STS-B数据集，详情见这里

# 训练
python train_unsup.py ./data/STS-B/cnsd-sts-train_unsup.txt

# 验证
python eval_unsup.py

模型	STS-B dev	STS-B test
hfl/chinese-bert-wwm-ext	0.3326	0.3209
simcse	0.7499	0.6909

与苏剑林的实验结果接近，BERT-P1是0.3465，SIMCSE是0.6904

A PyTorch implementation of unsupervised SimCSE

Related tags

Overview

A PyTorch implementation of unsupervised SimCSE

1. 用法

无监督训练

相似文本检索测试

STS-B数据集训练和测试

2. 参考

Owner

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

A code implementation of AC-GC: Activation Compression with Guaranteed Convergence, in NeurIPS 2021.

Code for the paper "Adversarial Generator-Encoder Networks"

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

[IEEE Transactions on Computational Imaging] Self-Gated Memory Recurrent Network for Efficient Scalable HDR Deghosting

EMNLP 2021 Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections

An efficient toolkit for Face Stylization based on the paper "AgileGAN: Stylizing Portraits by Inversion-Consistent Transfer Learning"

Measures input lag without dedicated hardware, performing motion detection on recorded or live video

"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

Multi-modal Vision Transformers Excel at Class-agnostic Object Detection

Scalable Graph Neural Networks for Heterogeneous Graphs

Collect some papers about transformer with vision. Awesome Transformer with Computer Vision (CV)

DiscoNet: Learning Distilled Collaboration Graph for Multi-Agent Perception [NeurIPS 2021]

TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios

This Deep Learning Model Predicts that from which disease you are suffering.

Using Python to Play Cyberpunk 2077

Code for "AutoMTL: A Programming Framework for Automated Multi-Task Learning"

Improving Contrastive Learning by Visualizing Feature Transformation, ICCV 2021 Oral

Extending JAX with custom C++ and CUDA code

MASS (Mueen's Algorithm for Similarity Search) - a python 2 and 3 compatible library used for searching time series sub-sequences under z-normalized Euclidean distance for similarity.