DistilBERT for Sentiment Analysis

Summary

• Introduction
• Parameters
• Examples
• Usage for developers
• Reference
• License
• Citation

Introduction

DistilBERT for Sentiment Analysis is an automated approach to analyze and classify emotions in text as positive or negative based on the expressed opinions.

Model Description: The model is a fine-tuned version of DistilBERT-base-uncased, specifically trained on the SST-2 dataset. It achieves an accuracy of 91.3 on the development set. For comparison, the Bert bert-base-uncased version achieves an accuracy of 92.7.

Parameters

Inputs

• input - (text): The input of the model is a sentence or a paragraph provided by the user that needs to be determined for its sentiment.

Output

• output - (text): The model provides predictions for each label, which are presented as probabilities, based on the two main labels: positive or negative.

Examples

input	output
Hello, my dog is cute.

Usage for developers

Please find below the details to track the information and access the code for processing the model on our platform.

Requirements

python~=3.10
torch~=2.0
transformers

Code based on AIOZ structure

from transformers import pipeline
import torch, os

... 
def do_ai_task(
        input: Union[str, Path],
        model_storage_directory: Union[str, Path],
        device: Literal["cpu", "cuda", "gpu"] = "cpu",
        *args, **kwargs) -> Any:
        
        model_id = os.path.abspath(model_storage_directory + "...")
        pipeline = pipeline(task="text-classification", model=model_id)
        result = model(input)
    return result

Reference

This repository is based on and inspired by Elliot Jiang's work We sincerely appreciate their generosity in sharing the code.

License

We respect and comply with the terms of the author's license cited in the Reference section.

Citation

@article{Sanh2019DistilBERTAD,
  title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
  author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf},
  journal={ArXiv},
  year={2019},
  volume={abs/1910.01108}
}

@inproceedings{socher-etal-2013-recursive,
    title = "Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank",
    author = "Socher, Richard  and
      Perelygin, Alex  and
      Wu, Jean  and
      Chuang, Jason  and
      Manning, Christopher D.  and
      Ng, Andrew  and
      Potts, Christopher",
    booktitle = "Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing",
    month = oct,
    year = "2013",
    address = "Seattle, Washington, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/D13-1170",
    pages = "1631--1642",
}