A Web-Based Multi-Modal AI Framework for Intelligent Content Moderation with Constructive Feedback
Article Sidebar
Main Article Content
Abstract
Traditional content moderation systems primarily rely on punitive measures that focus on removing offending or inappropriate content without offering users any form of educational feedback or guidance. Traditional content moderation systems primarily rely on punitive measures that focus on removing offending or inappropriate content without offering users any form of educational feedback or guidance. This lack of constructive interaction often leads to a poor user experience, leaving users frustrated and uninformed regarding the reasons behind content removal. Moreover, such a system fails to encourage long-term positive behavioural changes within online communities, as users do not gain insights on how to improve their conduct. This paper proposes a new constructive Artificial Intelligence (AI) intervention system that integrates a number of AI models in order to conceptualize content moderation as a limiting process rather than an educational process. The framework suggested in the paper combines HuggingFace's BERT-based toxicity analysis, SightEngine's image processing, and Google's Gemini large language model to provide end-to-end content moderation with construction of feedback. Based on comprehensive testing through user interaction analysis and comparative analysis, the study demonstrates the effectiveness of educational intervention in content moderation. The study advances the field by developing a novel approach to content moderation that puts user education and community development first while upholding safety regulations.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
[1] R. Gorwa, R. Binns, and C. Katzenbach, “Algorithmic content moderation: Technical and political challenges in the automation of platform governance,” Big Data and Society, vol. 7, no. 1, Jan. 2020, doi: 10.1177/2053951719897945.
[2] K. Klonick, “The new governors: The people, rules, and processes governing online speech,” Harvard Law Review, vol. 131, no. 6, pp. 1598–1670, Apr. 2018. [Online]. Available: https://harvardlawreview.org/2018/04/the-new-governors/
[3] S. Jhaver, I. Birman, E. Gilbert, and A. Bruckman, “Human-machine collaboration for content regulation: The case of reddit automoderator,” ACM Transactions on Computer-Human Interaction, vol. 26, no. 5, Jul. 2019, doi: 10.1145/3338243.
[4] D. Jurgens, E. Chandrasekharan, and L. Hemphill, “A just and comprehensive strategy for using NLP to address online abuse,” pp. 3658–3666, 2019, doi: 10.18653/v1/P19-1357.
[5] J. Seering, T. Wang, J. Yoon, and G. Kaufman, “Moderator engagement and community development in the age of algorithms,” New Media & Society, vol. 21, no. 7, pp. 1417–1443, 2019, doi: 10.1177/1461444818821316.
[6] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” Proceedings of the 2019 Conference of the North, pp. 4171–4186, 2019, doi: 10.18653/v1/n19-1423.
[7] J. N. Matias, “Preventing harassment and increasing group participation through social norms in 2,190 online science discussions,” Proceedings of the National Academy of Sciences, vol. 116, no. 20, pp. 9785–9789, 2019, doi: 10.1073/pnas.1813486116.
[8] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, and R. Kumar, “Predicting the type and target of offensive posts in social media,” Association for Computational Linguistics, 2019. doi: 10.18653/v1/N19-1144.
[9] R. Thoppilan et al., “LaMDA: Language models for dialog applications,” Feb. 2022, doi: 10.48550/arXiv.2201.08239.
[10] A. Arango, J. E. P. Perez, and B. Poblete, “Hate speech detection is not as easy as you may think,” Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 45–54, 2019, doi: 10.1145/3331184.3331262.
[11] M. Sap, D. Card, S. Gabriel, Y. Choi, N. A. Smith, and P. G. Allen, “The risk of racial bias in hate speech detection,” Association for Computational Linguistics, 2019 [Online]. Available: https://aclanthology.org/P19-1163/
[12] R. Bommasani et al., “On the opportunities and risks of foundation models,” Jul. 2022, [Online]. Available: http://arxiv.org/abs/2108.07258.
[13] A. Chowdhery et al., “PaLM: Scaling language modeling with pathways,” Oct. 2022, [Online]. Available: http://arxiv.org/abs/2204.02311.
[14] J. Lv, S. Cui, Y. Wang, G. Wu, Q. Zhang, and M. Huang, “Safe alone, unsafe together: Safeguarding against implicit toxicity when benign images combine,” Jul. 2026, [Online]. Available: http://arxiv.org/abs/2607.00576.
[15] A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” Jun. 2021, [Online]. Available: http://arxiv.org/abs/2010.11929.
[16] F. Hartvig, A. Egelund, and N. Elmqvist, “Trust me, I'm probably right: Investigating AI explanations for establishing appropriate trust,” in Proceedings of the 2026 International Conference on Advanced Visual Interfaces, Jun. 2026, pp. 1-9, doi: 10.1145/3811427.3811445.
[17] R. Anil et al., “PaLM 2 technical report,” Sep. 2023, doi: 10.48550/arXiv.2305.10403.
[18] OpenAI et al., “GPT-4 technical report,” Mar. 2024, doi: 10.48550/arXiv.2303.08774.
[19] E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the dangers of stochastic parrots: Can language models be too big?,” in FAccT 2021 - Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, Inc, pp. 610–623, Mar. 2021, doi: 10.1145/3442188.3445922.
[20] J. Welbl et al., “Challenges in Detoxifying Language Models,” Sep. 2021, doi: 10.48550/arXiv.2109.07445.
[21] S. Pichai and D. Hassabis, "Introducing Gemini: Our largest and most capable AI model," Google Blog, Dec. 6, 2023. [Online]. Available: https://blog.google/innovation-and-ai/technology/ai/google-gemini-ai/#sundar-note. [Accessed: Jun. 23, 2026].
[22] H. Touvron et al., “Llama 2: Open foundation and fine-tuned chat models,” Jul. 2023, [Online]. Available: http://arxiv.org/abs/2307.09288.
[23] Y. Wang et al., “Self-Instruct: Aligning language models with self-generated instructions,” Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13484–13508, 2023, doi: 10.18653/v1/2023.acl-long.754.
[24] J. Wei et al., “Chain-of-thought prompting elicits reasoning in large language models,” Jan. 2023, [Online]. Available: http://arxiv.org/abs/2201.11903.
[25] S. Min et al., “Rethinking the role of demonstrations: What makes in-context learning work?,” Oct. 2022, [Online]. Available: http://arxiv.org/abs/2202.12837
[26] T. B. Brown et al., “Language models are few-shot learners,” 2020. [Online]. Available: http://arxiv.org/abs/2005.14165
[27] J. Liu, D. Shen, Y. Zhang, B. Dolan, L. Carin, and W. Chen, “What Makes Good In-Context Examples for GPT-3?,” Jan. 2021, [Online]. Available: http://arxiv.org/abs/2101.06804.
[28] L. Ouyang et al., “Training language models to follow instructions with human feedback,” Mar. 2022, doi: 10.48550/arXiv.2203.02155.
[29] J. Hoffmann et al., “Training compute-optimal large language models,” Mar. 2022, [Online]. Available: http://arxiv.org/abs/2203.15556.
[30] L. Dixon, J. Li, J. Sorensen, N. Thain, and L. Vasserman, “Measuring and mitigating unintended bias in text classification,” in AIES 2018 - Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, Association for Computing Machinery, Inc, pp. 67–73, Dec. 2019, doi: 10.1145/3278721.3278729.