RoBERTa Large

RoBERTa Large is a 355M-parameter English masked-language model pretrained by Meta's FacebookAI lab, designed as a robust text encoder for downstream NLP tasks such as classification, named entity recognition, and question answering. It improves on BERT-style pretraining through dynamic masking, removal of the next-sentence-prediction objective, larger training data, and larger batch sizes. Source: supabase.