Year
Month
(Preprint) From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network
Yuxin Wang 王裕鑫 ¹, Hongtao Xie 谢洪涛 ¹, Shancheng Fang ¹, Jing Wang ², Shenggao Zhu ², Yongdong Zhang 张勇东 ¹
¹ University of Science and Technology of China
中国科技大学
² Huawei Cloud & AI
华为云人工智能
arXiv, 2021-08-22
Abstract

In this paper, we abandon the dominant complex language model and rethink the linguistic learning process in the scene text recognition. Different from previous methods considering the visual and linguistic information in two separate structures, we propose a Visual Language Modeling Network (VisionLAN), which views the visual and linguistic information as a union by directly enduing the vision model with language capability. Specially, we introduce the text recognition of character-wise occluded feature maps in the training stage. Such operation guides the vision model to use not only the visual texture of characters, but also the linguistic information in visual context for recognition when the visual cues are confused (e.g. occlusion, noise, etc.).

As the linguistic information is acquired along with visual features without the need of extra language model, VisionLAN significantly improves the speed by 39% and adaptively considers the linguistic information to enhance the visual features for accurate recognition. Furthermore, an Occlusion Scene Text (OST) dataset is proposed to evaluate the performance on the case of missing character-wise visual cues. The state of-the-art results on several benchmarks prove our effectiveness.
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_1
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_2
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_3
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network_4
  • All-optical object identification and three-dimensional reconstruction based on optical computing metasurface
  • Dingyu Xu, Wenhao Xu, Qiang Yang, Wenshuai Zhang, Shuangchun Wen, Hailu Luo,
  • Opto-Electronic Advances
  • 2023-12-12
  • Applications of lasers: A promising route toward low-cost fabrication of high-efficiency full-color micro-LED displays
  • Shouqiang Lai, Shibiao Liu, Zilu Li, Zhening Zhang, Zhong Chen, Rong Zhang, Hao-Chung Kuo, Tingzhu Wu
  • Opto-Electronic Science
  • 2023-12-06
  • Light-stimulated adaptive artificial synapse based on nanocrystalline metal-oxide film
  • Igor S. Balashov, Alexander A. Chezhegov, Artem S. Chizhov, Andrey A. Grunin, Konstantin V. Anokhin, Andrey A. Fedyanin
  • Opto-Electronic Science
  • 2023-12-06
  • Integrated photonic convolution acceleration core for wearable devices
  • Baiheng Zhao, Junwei Cheng, Bo Wu, Dingshan Gao, Hailong Zhou, Jianji Dong
  • Opto-Electronic Science
  • 2023-11-28
  • Optical trapping of optical nanoparticles: Fundamentals and applications
  • Fengchan Zhang, Pablo Camarero, Patricia Haro-González, Lucía Labrador-Páez, Daniel Jaque
  • Opto-Electronic Science
  • 2023-11-15
  • Pattern recognition in multi-synaptic photonic spiking neural networks based on a DFB-SA chip
  • Yanan Han, Shuiying Xiang, Ziwei Song, Shuang Gao, Xingxing Guo, Yahui Zhang, Yuechun Shi, Xiangfei Chen, Yue Hao
  • Opto-Electronic Science
  • 2023-11-15
  • Ferroelectrically modulate the Fermi level of graphene oxide to enhance SERS response
  • Mingrui Shao, Chang Ji, Jibing Tan, Baoqiang Du, Xiaofei Zhao, Jing Yu, Baoyuan Man, Kaichen Xu, Chao Zhang, Zhen Li
  • Opto-Electronic Advances
  • 2023-11-15
  • Photonic integrated neuro-synaptic core for convolutional spiking neural network
  • Shuiying Xiang, Yuechun Shi, Yahui Zhang, Xingxing Guo, Ling Zheng, Yanan Han, Yuna Zhang, Ziwei Song, Dianzhuang Zheng, Tao Zhang, Hailing Wang, Xiaojun Zhu, Xiangfei Chen, Min Qiu, Yichen Shen, Wanhua Zheng, Yue Hao
  • Opto-Electronic Advances
  • 2023-11-15
  • Paper-based WS₂ photodetectors fabricated by all-dry techniques
  • Francesco Pieri, Gianluca Fiori
  • Opto-Electronic Advances
  • 2023-11-15
  • Spatiotemporal hemodynamic monitoring via configurable skin-like microfiber Bragg grating group
  • Hengtian Zhu, Junxian Luo, Qing Dai, Shugeng Zhu, Huan Yang, Kanghu Zhou, Liuwei Zhan, Biao Xu, Ye Chen, Yanqing Lu, Fei Xu
  • Opto-Electronic Advances
  • 2023-11-15
  • Physics-data-driven intelligent optimization for large-aperture metalenses
  • Yingli Ha, Yu Luo, Mingbo Pu, Fei Zhang, Qiong He, Jinjin Jin, Mingfeng Xu, Yinghui Guo, Xiaogang Li, Xiong Li, Xiaoliang Ma, Xiangang Luo
  • Opto-Electronic Advances
  • 2023-11-15
  • Metasurfaces for near-eye display applications
  • Yan Li, Xiaojin Huang, Shuxin Liu, Haowen Liang, Yuye Ling1, Yikai Su
  • Opto-Electronic Science
  • 2023-10-31



  • NetGraph: An Intelligent Operated Digital Twin Platform for Data Center Networks                                IGNNITION: fast prototyping of graph neural networks for communication networks
    About
    |
    Contact
    |
    Copyright © PubCard