Lightweight Multimodal Anomaly Detection Based on GhostNet and FiLM
DOI:
https://doi.org/10.54097/1p6pw376Keywords:
Abnormal behavior detection, multimodal fusion, GhostNet, FiLM, lightweight modelAbstract
To address the trade-off between detection accuracy and deployment efficiency in surveillance-based anomaly detection, we propose a lightweight multimodal abnormal behavior detection method based on GhostNet and FiLM fusion (G-FLM). GhostNet is employed as a lightweight visual backbone to reduce computational redundancy, while audio features are introduced to generate FiLM-based channel-wise affine parameters to generate FiLM-based channel-wise affine parameters for conditional recalibration of video features, thereby achieving effective audio-video information fusion with limited additional overhead. Experimental results on the XD-Violence dataset show that, under the same experimental configuration, G-FLM achieves an mAP of 0.7392. Compared with multimodal baselines such as Two-Stream and I3D, the mAP is improved by 5.6% and 18.0%, respectively. Precision reaches 0.8163 and F1 reaches 0.7260, both outperforming the comparison methods. At the same time, the model has only 3.63M parameters, requires only 1.9054 GFLOPs, and achieves an inference latency of 30.53 ms/batch, achieving higher detection accuracy with substantially lower computational overhead than the compared baselines. The results indicate that G-FLM achieves a good overall balance between detection accuracy and deployment efficiency, and provides a feasible lightweight solution for multimodal abnormal behavior detection in resource-constrained scenarios.
Downloads
References
[1] Sultani, W., Chen, C., & Shah, M. (2018). Real world anomaly detection in surveillance videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 6479 6488). Salt Lake City, USA.
[2] Wu, P., Liu, J., Shi, Y., et al. (2020). Not only look, but also listen: Learning multimodal violence detection under weak supervision. In European Conference on Computer Vision (pp. 322 339). Glasgow, UK.
[3] Tian, Y., Pang, G., Chen, Y., et al. (2021). Weakly supervised video anomaly detection with robust temporal feature magnitude learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 4975 4986). Montreal, Canada.
[4] Zaheer, M. Z., Mahmood, A., Astrid, M., & Lee, S. I. (2020). CLAWS: Clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection. In European Conference on Computer Vision. Glasgow, UK.
[5] Mao, J. (2024). Research on weakly supervised violence detection technology based on modality fusion (Master’s thesis). Xidian University, China. (In Chinese)
[6] Wu, P., Li, W., Guo, F., et al. (2025). Video anomaly detection based on feature enhancement and modal interaction. Journal of Computer Aided Design & Computer Graphics, 37(3), 407 413. (In Chinese)
[7] Simonyan, K., & Zisserman, A. (2014). Two stream convolutional networks for action recognition in videos. In Advances in Neural Information Processing Systems (pp. 568 576). Montreal, Canada.
[8] Tran, D., Bourdev, L., Fergus, R., et al. (2015). Learning spatiotemporal features with 3D convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 4489 4497). Santiago, Chile.
[9] Carreira, J., & Zisserman, A. (2017). Quo vadis, action recognition? A new model and the Kinetics dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 6299 6308). Honolulu, USA.
[10] Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020). GhostNet: More features from cheap operations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1580 1589). Seattle, USA. arXiv:1911.11907.
[11] Liu, C., Duan, X., & Xie, B. (2022). Improvement of ECO target tracking algorithm based on GhostNet convolution features. Laser Technology, 46(2), 239 247. (In Chinese)
[12] Tang, Y., Han, K., Guo, J., Xu, C., Xu, C., & Wang, Y. (2022). GhostNetV2: Enhance cheap operation with long range attention. In Advances in Neural Information Processing Systems. New Orleans, USA. arXiv:2211.12905.
[13] Putra, A. K., Santika, S. P., Peranginangin, E., & Girsang, A. S. (2025). Lightweight YOLO11 with GhostNet and CBAM for real time fire detection. In 2025 IEEE International Symposium on Future Telecommunication Technologies (pp. 316 321). Lombok, Indonesia. https://doi.org/10.1109/SOFTT67007.2025.11213346
[14] Perez, E., Strub, F., de Vries, H., Dumoulin, V., & Courville, A. (2018). FiLM: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 3942 3951). New Orleans, USA. arXiv:1709.07871.
[15] Lin, T. Y., Goyal, P., Girshick, R., et al. (2017). Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2980 2988). Venice, Italy.
[16] Srivastava, N., Hinton, G., Krizhevsky, A., et al. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1), 1929 1958.
[17] Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. In International Conference on Learning Representations. New Orleans, USA.
[18] Howard, A., Sandler, M., Chen, B., et al. (2019). Searching for MobileNetV3. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1314 1324). Seoul, South Korea.
[19] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770 778). Las Vegas, USA.
[20] Chen, X. (2021). Research on fast object finding technology in video (Master’s thesis). Xidian University, China. (In Chinese)
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Advanced Engineering and Technology Research
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
