A Scalable Transformer-Based Framework for Sentiment Analysis and Public Opinion Mining Across Heterogeneous Social Media Platforms

Abstract

Social media platforms generate more than 500 million posts every day, creating a constantly changing record of public opinion about political events, public health crises, brands, and social movements. Although this data is extremely valuable, turning raw social media text into clear and useful opinion insights is still difficult because of three main challenges. First, different platforms such as Twitter/X, Reddit, YouTube, and Facebook use different writing styles, vocabulary, post lengths, and noise patterns. Second, social media language often includes code- switching, sarcasm, implicit sentiment, and platform-specific slang, which makes it hard for traditional lexicon- based or single-dataset trained models to understand the real meaning. Third, analyzing millions of posts in real time requires highly scalable computational systems. To address these challenges, this paper introduces the Cross-Platform Sentiment and Opinion Intelligence Network (CPSOIN), a unified transformer-based framework. CPSOIN combines three key components: Platform-Adaptive Fine-Tuning (PAFT) applied to a RoBERTa-large model to handle differences between platforms, a Sarcasm-Aware Contextual Encoding (SACE) module that detects sarcasm using incongruity between sentence meaning and knowledge-based expectations, and a Distributed Streaming Inference Engine (DSIE) built with Apache Kafka and ONNX Runtime. This system can process up to 47,300 posts per second with latency below 80 milliseconds on standard hardware. The framework is evaluated on five datasets: SemEval-2017 Task 4, SentiRaama, Reddit SARC 2.0, Twitter COVID-19 Sentiment, and a new Indian General Election 2024 opinion dataset. Results show that CPSOIN improves weighted F1 scores by 3.1–8.7 percentage points compared to strong baseline models, reduces sarcasm misclassification by 34.2%, and achieves 4.1× higher inference throughput than standard PyTorch-based systems at production-scale data volumes.