Korean stereoypte detector with TUNiB-Electra and K-StereoSet

Last update: Feb 18, 2022

Related tags

Text Data & NLP Stereotype-Detector

Overview

Korean Stereotype Detector

Korean stereotype sentence classifier using K-StereoSet with TUNiB-Electra

Web demo

you can test this model easily in demo page
LINK: https://share.streamlit.io/newfull5/stereotype-detector/demo.py

Dataset

There are 12,688 human-labeled comments in total.
They ar splitted into 10,688 train set, 1000 test set and 1000 valid set

stereotype, anti-stereotype, unrelated, profession, race, gender, religion, comment
1.0,    0.0,    0.0,    1.0,    0.0,    0.0,    0.0,    축구선수의 오랜 경력의 전형적인 결과는 부상을 입은 발목과 두개골이다.
1.0,    0.0,    0.0,    0.0,    1.0,    0.0,    0.0,    콜롬비아 사람들은 모두 마약 카르텔에 소속되어 있다.
0.0,    1.0,    0.0,    0.0,    1.0,    0.0,    0.0,    그는 무기를 소지한 적이 없다.
0.0,    1.0,    0.0,    0.0,    0.0,    1.0,    0.0,    그는 주말에만 지루할 수 있는 일을 한다.

Detail

	stereotype	anti-stereotype	unrelated	profession	race	gender	religion	Total
Train	3,550	3,556	3,581	4,140	4,896	1,268	383	10,688
Valid	341	347	312	410	435	110	45	1,000
Test	334	324	336	361	483	113	43	1,000

Score

	precision	recall	F1
stereotype	0.814	0.601	0.691
anti-stereotype	0.894	0.509	0.648
unrelated	0.872	0.870	0.871
profession	0.943	0.711	0.811
race	0.787	0.907	0.843
gender	0.639	0.836	0.724
religion	0.724	1.0	0.840
total (macro score)	0.810	0.776	0.775

Usage

training

python3 train.py --model_name tunib/electra-ko-base \
                 --data_dir YOUR_PATH \
                 --batch_size BATCH_SIZE \

threshold optimizing

python3 threshold.py --model_name tunib/electra-ko-base \
                     --data_dir YOUR_CKPT_DIR_PATH \
                     --file_path YOUR_CKPT_FILE_NAME \
                     --batch_size BATCH_SIZE \
                     --data_path TEST_DATA_PATH

test

python3 score.py --model_name tunib/electra-ko-base \
                 --data_dir YOUR_CKPT_DIR_PATH \
                 --file_path YOUR_CKPT_FILE_NAME \
                 --batch_size BATCH_SIZE \
                 --data_path TEST_DATA_PATH

Korean stereoypte detector with TUNiB-Electra and K-StereoSet

Related tags

Overview

Korean Stereotype Detector

Web demo

Dataset

Detail

Score

Usage

Owner

Sae_Chan_Oh

Text Analysis & Topic Extraction on Android App user reviews

Fast, DB Backed pretrained word embeddings for natural language processing.

BPEmb is a collection of pre-trained subword embeddings in 275 languages, based on Byte-Pair Encoding (BPE) and trained on Wikipedia.

Espresso: A Fast End-to-End Neural Speech Recognition Toolkit

Use the state-of-the-art m2m100 to translate large data on CPU/GPU/TPU. Super Easy!

Code for the paper "BERT Loses Patience: Fast and Robust Inference with Early Exit".

Rhythm-Finder is a unsupervised ML driven python powered web-application that can find the songs that suits you.

Large-scale open domain KNOwledge grounded conVERsation system based on PaddlePaddle

An easy-to-use Python module that helps you to extract the BERT embeddings for a large text dataset (Bengali/English) efficiently.

TPlinker for NER 中文/英文命名实体识别

pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.

OceanScript is an Esoteric language used to encode and decode text into a formulation of characters

This is a MD5 password/passphrase brute force tool

In this Notebook I've build some machine-learning and deep-learning to classify corona virus tweets, in both multi class classification and binary classification.

Continuously update some NLP practice based on different tasks.

Telegram AI chat bot written in Python using Pyrogram

Code for paper "Which Training Methods for GANs do actually Converge? (ICML 2018)"

BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese

KR-FinBert And KR-FinBert-SC

Uncomplete archive of files from the European Nopsled Team