Text Generation with SmolLM-135M

Summary

• Introduction
• Parameters
• Examples
• Usage for developers
• Reference
• License
• Citation

Introduction

Text Generation task is a fundamental application of artificial intelligence where machine learning models are employed to generate new text based on training data. In this domain, the SmolLM (Small Language Model) has emerged as a popular tool for automatic text generation. SmolLM is a compact language model typically built on recurrent neural network (RNN) or long short-term memory (LSTM) architectures. Despite its small size, SmolLM is capable of producing natural and structured text.

By leveraging rich training data, SmolLM can learn language structure, vocabulary, and context to generate text in a coherent and continuous manner. This tool finds applications in various scenarios, including automated blog writing, image captioning, or even short story creation. SmolLM not only saves time but also enhances creativity and efficiency in text content generation.

SmolLM is a series of state-of-the-art small language models available in three sizes: 135M, 360M, and 1.7B parameters. These models are built on Cosmo-Corpus, a meticulously curated high-quality training dataset. Cosmo-Corpus includes Cosmopedia v2 (28B tokens of synthetic textbooks and stories generated by Mixtral), Python-Edu (4B tokens of educational Python samples from The Stack), and FineWeb-Edu (220B tokens of deduplicated educational web samples from FineWeb). SmolLM models have shown promising results when compared to other models in their size categories across various benchmarks testing common sense reasoning and world knowledge. For comprehensive details regarding training, benchmarks, and performance, please consult the author's complete blog post.

Training

Model

• Architecture: For architecture detail, see the blog post.
• Pretraining steps: 500k
• Pretraining tokens: 1T
• Precision: bfloat16

Hardware

• GPUs: 64 H100

Software

• Training Framework: Nanotron

Parameters

Inputs

• prompt - (text): The input for this task is a textual prompt, which can be a few words or a complete sentence, serving as a guideline for the desired text generation.
• lenght - (number-integer): The variable represents the maximum length or size constraint for the generated output. It ensures that the model does not generate excessively long or unwieldy responses.

Output

• output - (text): The output of model is a generated text that follows the context and style of the given prompt. The model utilizes its understanding of language patterns, grammar, and semantics to generate coherent and meaningful text.

Examples

prompt	lenght	output
Once upon a time	20	Once upon a time, in a small town named Harmonyville, lived two best friends

Usage for developers

Please find below the details to track the information and access the code for processing the model on our platform.

Requirements

python~=3.10
torch~=2.0.0
transformers
os

Code based on AIOZ structure

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch, os

...

def do_ai_task(
        prompt: Union[str, Path],
        lenght: Union[int, int],
        model_storage_directory: Union[str, Path],
        device: Literal["cpu", "cuda", "gpu"] = "cpu",
        *args, **kwargs) -> Any:
        
        model_id = os.path.abspath(model_storage_directory + /...)
        tokenizer = AutoTokenizer.from_pretrained(model_id)
        model = AutoModelForCausalLM.from_pretrained(model_id).to(device)
        
        inputs = tokenizer.encode(prompt, return_tensors="pt").to(device)
        outputs = model.generate(inputs)
        outputs = tokenizer.decode(outputs[0])
        
    return outputs

Reference

This repository is based on and inspired by Hugging Face TB Research's work. We sincerely appreciate their generosity in sharing the code.

License

We respect and comply with the terms of the author's license cited in the Reference section.

Citation

@misc{allal2024SmolLM,
      title={SmolLM - blazingly fast and remarkably powerful}, 
      author={Loubna Ben Allal and Anton Lozhkov and Elie Bakouch and Leandro von Werra and Thomas Wolf},
      year={2024},
}