LETTERNET: A CNN based Encoder/Decoder for Visual Data Transfer beyond Barcodes

LETTERNET: A CNN based Encoder/Decoder for Visual Data Transfer beyond Barcodes Oliver Urbann Fraunhofer IML 44227 Dortmund [email protected] Piet Br¨ ommel Fraunhofer IML 44227 Dortmund [email protected] Abstract Machine-readable static data can be encoded in a variety of ways. While barcodes, such as QR codes, are a popular form, human writing utilizing OCR can also be used, but this is not optimal if the goal is machine readability. Thus, the question arises, what form of representation would a machine learning algorithm invent to be able to transmit data over a static medium such as paper? In this work, we will contribute a CNN that learns to generate an image from characters in such a way that it can be decoded back to the original characters by a decoder under given degradations. These three components, encoder, degradations, and decoder, are trained in an end-to-end fashion. In the evaluation, we show how the generated images appear for different degradations, and also show that this form is more robust than QR codes decoded by ZBar for different string lengths. Furthermore, we show that even ZBar with prior removal of blur and noise by a state-of-the-art method performs worse than the proposed LETTERNET. Finally, our sim-to-real tests show that the sim-to-real gap is small even though the training is based on purely artificial datasets and degradations. 1. Introduction Encoding small chunks of data that can be printed or otherwise displayed statically, e.g. on paper, is not only important for end users who use their cell phones to read URLs, for example. It is also important in an industrial context. In logistics, for example, paper-based markers are still less expensive than RFID codes and can still be found on almost every pallet load. Reading these paper tags is usually done manually. However, when autonomous vehicles attempt to identify the pallet using a mounted camera, clear, low-interference image capture with good resolution is either impossible or very costly. There are many variations of bar codes with different diFigure 1. The word LETTERNET as ”written” by a LargeNetEmbd CNN (encoder part) trained for interferences B(0.4), G(40), N(0.4). Figure 2. Application of machine-readable codes in logistics. mensions. Most of them share the idea of being easily readable by machines, so they encode the data with different bar widths or by setting black, white or a color to encode binary data. In addition, error correction codes can be applied, such as the Reed-Solomon code. In contrast, human scripts are rooted in the tools available (e.g., hammer and chisel, quill and pen) and the way humans perceive objects and thus write. This results in scripts with thin lines and where the form encodes the data. Redundancy is provided by greater differentiation between the letters of the font. For example, license plates use optimized fonts, and historical computers displayed or printed the zero with a slash to distinguish it from the letter O for better readability at the lower resolutions of the day. 1.1. LETTERNET With this work, we contribute a study of visual representations as generated by a novel convolutional neural net- Encoder Degradation Decoder ”4MWAJBBVU” Embedding 9·36 Reshape with Zero Padding 64 16 Tr. Convolution (tanh) Upsampling 128 32 Tr. Convolution (tanh) Upsampling 256 64 Tr. Convolution (tanh) Upsampling 512 128 Tr. Convolution (mapping) Brightness Bσb Blur Gks,σg Noise Nµ,σn Convolution (tanh) 512 128 Max-Pooling Convolution (tanh) 256 64 Max-Pooling Convolution (tanh) 128 32 Max-Pooling Convolution (tanh) 64 16 Dense (sigmoid) 323 ”4MWAJBBVU” Figure 3. The proposed LETTERNET for encoding, degradation and decoding in one pipeline for training. In a practical application, the coder would generate an image representing the input data, here a nine-character string for each network output. This image could be captured by a camera, and the decoder would then reconstruct the original payload data. The typical degradation of the image due to the inclusion of a camera is simulated here by the degradation block. Note that not all layers are listed for clarity, and the relative scaling does not reflect the true proportions. In brackets are the activations belonging to the layer, some of which are custom. work (CNN). This CNN is divided into three parts: an encoder to generate an image from given data, connected to non-trainable degradation layers, e.g. noise and blur, and a decoder to read the data from the image. To be comparable to human writing and barcodes, we decided to match the shape of the data to be encoded. Thus, our symbol data set consists of uppercase letters and digits. Similar to a CNN for classification, our CNN uses a fixed size of input and output data. This can be a single symbol, as in human writing, or multiple symbols, as in QR codes. In this work, we decided to use 9 symbols at a time. This way we are comparable to the smallest QR codes and can encode the string LETTERNET at once. Note that longer strings are encoded like in human writing by concatenating several visual representations or by using different versions. The evaluation shows this in detail for LETTERNET and QR Codes. In this work we are interested in the following questions: 1. How would such a picture look like? 2. Which CNN designs lead to good results under different degradations? 3. How does it perform compared to another form of machine-readable encodings? 4. Do these results also apply to longer strings? 5. How big is the sim-to-real gap? It is known that CNN have their background in biological vision [7]. Thus, we expect it to share properties with human writing, such as achieving redundancy through more differentiated shapes. However, we would not expect an ML approach to produce a font that consists entirely of thin lines, as these are derived from the tools available for human writing. Also, we expect a significant sim-to-real gap, as our degradations are artificial. 1.2. Contribution To be specific, our contribution is in particular 1. a new class of CNN application as visual data encoder, besides other existing types like autoencoders, detection, etc., 2. a novel design of a CNN, (a) which adopts parts of autoencoders but applies them in reverse, (b) novel degradation layers that implement known augmentations into a layer usable for training, (c) integration of an embedding layer with special initialization (Perlin) that has not been used for image processing so far, 3. evaluation of different CNN configurations and string lengths, 4. comparison of the LETTERNET performance with QR codes, also with preprocessing, 5. evaluation of the artificially trained network on real images. 1.3. Related Work To the best of our knowledge, end-to-end learned visual data transfer has not been studied before. However, already considered was the generation of human fonts by GANs, especially for Chinese [6,4]. Nevertheless, it is known to be a relevant application area that is currently dominated by barcodes. Research is therefore aimed at being able to robustly recognize and read barcodes, and QR codes in particular, even under disruptive conditions. The open source library ZBar1is well-known for its good reading performance even under difficult conditions [11] and is therefore a popular method to evaluate the performance of preprocessing steps. Note that although the library has not been updated for some time now, it is still used for comparisons in recent publications and is considered state-of-the-art [1]. We thus also utilize this library in our evaluation. Jain et al. [3] give a recent overview of several approaches to improve the readability of barcodes by applying different preprocessing steps. Wudhikarn et al. [13] give a similar overview, especially for approaches based on deep learning. Accordingly, a popular application of deep learning is the detection of QR codes [2], especially from strongly different points of view [5]. Besides improved detection also reading the QR code can be improved by applying preprocessing steps based on CNN, e.g. for deblurring [9], or by applying a Generative Adversative Network [12]. Gennip et al. [10] propose an approach not based on deep learning but that can also denoise the image. Furthermore, they evaluate how this improves the readability utilizing ZBar. While the evaluations of the previous approaches do not give such a detailed evaluation, this readability test with ZBar is also valuable for our evaluation, as we will explain in Sec. 3. 2. Methodology In this section we explain the details of the CNN depicted in Fig. 3, which we call LETTERNET, and give details about the training process. The overall structure of the deep neural network LETTERNET is inspired by an autoencoder, which maps data (often images) to a feature vector [15]. The input to LETTERNET is a string of characters from the ASCII set, namely the letters A-Z and the digits 0-9. The data is scaled to an image with a height of 128 and a width of 512. This forms the encoder part of the net and is explained in the next section. In a real-world application this image could then be printed or otherwise displayed to a camera that takes a picture for the decoder part of LETTERNET that translates this back to the original input string. 1https://zbar.sourceforge.net We train LETTERNET in an end-to-end fashion with the goal of studying what kind of representation or ”writing” an artificial intelligence would learn under given circumstances. Thus, the degradation normally caused by taking photographs must be artificially reproduced. Due to the optimization methods applied to neural networks, i.e. gradient descent, the degradation must be differentiable. For the scope of this work, we therefore use layers that are usually utilized for image augmentation. However, as we will discuss in more detail in Sec. 2.2, most of them are selfimplemented to meet all requirements. In this section we discuss LETTERNET as depicted in Fig. 3. For a kind of ablation study, we are also investigating variations of this network to better understand individual design decisions. These derivates will be introduced in Sec. 3. 2.1. Encoder/Decoder As introduced before, the input of the encoder part of LETTERNET is a string of letters A-Z and digits 0-9. This input must be of fixed length, and we chose 9 for this work because it is a squared number (used later for derivatives of this net) and this encoder can write its own name. Longer strings can be used as introduced in Sec. 1.1. Each letter is represented by an integer within the range [0,35] (the number of letters and digits here). The first layer, the Embedding, is known from Natural Language Processing where it is popular for representing words. Our application is similar, we want to represent characters and enable all possible graphical representations independent of ASCII code or any other data representation format. The Embedding layer enables this decoupling. In simple terms, it is a kind of array that outputs the corresponding data depending on the addressing by the input. We interpret this output as a small image for each input character. As initializer we apply 2D Perlin Noise [8], which is a gradient noise intended for application in computer graphics as a texture generator. For each letter, we chose an output dimension as small as possible to not predefine a specific look for a character, but long enough to enable the optimizer to make all letters distinguishable. Experiments have shown that a length of 6·6 = 36 for each character is a good choice. Thus, the output of the Embedding layer is a one-dimensional array of length 9·6·6. The next four layers are summarzied as one layer in Fig. 3. They reshape the output of the Embedding layer to transform the semantics of a data array to a two-dimensional image. Additionally, a zero padding is applied to add a margin for a later translational degradation, see Sec. 2.2. The following block is a Deconvolution with a filter of shape 6×6, tanh activation and an Upsampling layer. This is repeated twice with a filter size of 3×3and finished with another Deconvolutional layer with another 3×3filter. This last layer has a custom activation function derived from tanh as another factor to ensure image semantics: ϕ(v) = tanh(v)+1 2(1) This way the output of the encoder is limited to the range [0,1] with a height of 128 and a width of 512. As usual with autoencoders, the decoder backend is built in reverse, except that the activation functions are all tanh. The decoder head consists of a single Dense layer with a preceding reshape and Sigmoid activation function. The output shape is different from the input of the encoder. We decided to use a one-hot encoding for each character, thus the dimension is 9×36. 2.2. Degradations To simulate the degradations of real photography we selected four different types known for dataset augmentation: Random changes in brightness, Gaussian blur, Gaussian noise and random translation. Brightness and Noise In an industrial environment, especially in warehouses, lightning is artificial and thus constant. However, to avoid motion blur, the camera is set to a fixed low exposure time. Therefore, overand underexposure cannot be avoided and is simulated by layer Bσbfor random brightness, a common method for augmenting image data. The change in brightness is calculated by adding a random value x∼ N (µ, σ2 b)to all pixels in the batch, where σbdepends on the experiment parameterization, see Sec. 3. Note that xis redetermined for each batch during training to ensure that the net is able to work with different brightness conditions. Then all pixel values are clipped to the range [0,1]. The algorithm for the Nµ,σnlayer, which adds Gaussian noise, is quite similar, except that the random value is different for all pixels and is sampled separately for each pixel. Blur As explained before, an underexposed shot is accepted for reduced motion blur. We therefore simulate a 2D Gaussian blur function Gks,σgonly. In our implementation, the kernel size ksis selectable and σgcan be fixed but also randomly chosen, which is then denoted by Gks. During training the random value xis sampled once for each batch and then utilized to determine σg: σg(x) = x·(ks−1) 6, x ∼ U(0.3,1) (2) The layer is based on a 2D Depthwise Convolutional layer with the kernel matrix kgiven by: hg(i, j) = e −(i2+j2) 2σ2 g(3) kh,w,c =1 PiPjhg ·hg(i, j)(4) h={1, ..., ks}, w ={1, ..., ks}, c ={1, ..., C}(5) Note that the weight calculation is, in contrast to usual layers, implemented as a TensorFlow Graph as well, to enable sampling new random values during the training. Translation Before the contents of a barcode can be read, it must be detected first. However, since this is an error-prone process, e.g. QR codes have markers so that positioning errors can be corrected. A similar problem can be expected here, so the learning process should be able to simulate such errors. The Translation layer T∆therefore should be able to move the content about a factor of ∆randomly. While redundancy of data could also be an interesting goal, it is not within the scope of this work. Thus, the content should not be cropped, but should always be fully visible. To ensure this, the image is padded after the Embedding layer, so that T∆can be based on the Keras layer ”RandomTranslation”. 2.3. Training The training dataset is generated randomly and consists of 100.000 strings containing 9 characters each, with 10.000 images reserved for validation2. Characters are encoded as integers as input to the Embedding layer and one-hot encoded as output. Training is performed for 1000 epochs with batch size 10.000 and Adam optimizer with binary cross-entropy loss function. As mentioned in the introduction, human writing is influenced by many aspects like the utilized tools and human cognition. We therefore pre-train the backend of LETTERNET with images depicting natural objects for 20 epochs with the Fashion MNIST [14] dataset, again with Adam optimizer and binary cross-entropy loss function. This does not specify a scripture, but it supports the idea of human perception on which human writing is based. 3. Evaluation In the evaluation we want to first compare a few different CNN designs to investigate the influence of the Embedding layer and other design choices. Even though we don’t propose individual components, we want to investigate some in an ablation study fashion. Especially interesting is the influence of the degradation layers, if these are sufficient for our 2Due to the simplicity of this dataset, that can easily be generated, we refrain from publishing it. Table 1. Readability of strings from test dataset for various degradations. Each cell shows the results of all three passes and an example image after applying degradations (QR) or before (others). Note that the size of the image does not reflect its resolution. QR MediumNet LargeNetDense LargeNetEmbd 1.0, 1.0, 1.0 0.94, 0.22, 0.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.05,G51.0, 1.0, 1.0 0.97, 0.73, 0.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.1,G10 1.0, 1.0, 1.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.2,G20 1.0, 1.0, 1.0 1.0, 1.0, 0.95 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.25,G30 1.0, 1.0, 1.0 1.0, 0.94, 0.95 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.5,G40 1.0, 1.0, 0.99 0.61, 0.68, 0.78 0.8, 0.92, 0.95 0.95, 0.95, 0.94 B0.5,G60 0.0, 0.01, 0.0 0.0, 0.35, 0.1 0.65, 0.56, 0.67 0.95, 0.94, 0.95 B0.05,G5,N0.05 1.0, 1.0, 1.0 0.0, 1.0, 0.94 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.1,G10,N0.10.0, 0.0, 0.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.15,G15,N0.15 0.0, 0.0, 0.0 0.0, 1.0, 0.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.2,G20,N0.20.0, 0.0, 0.0 0.0, 1.0, 0.0 1.0, 1.0, 1.0 1.0, 1.0, 1.0 B0.4,G40,N0.40.0, 0.0, 0.0 0.0, 0.0, 0.0 0.73, 0.58, 0.68 0.97, 0.96, 0.96 B0.1,G10,N0.1,T0.05 0.71, 0.64, 0.68 0.0, 0.0, 0.0 0.99, 0.99, 1.0 0.9, 0.88, 0.88 B0.15,G15,N0.15,T0.05 0.08, 0.1, 0.06 0.0, 0.0, 0.0 0.96, 0.96, 0.87 0.88, 0.86, 0.89 B0.2,G20,N0.2,T0.10.0, 0.0, 0.0 0.0, 0.0, 0.0 0.0, 0.0, 0.0 0.0, 0.0, 0.0 Table 2. Results of the comparison of LargeNetEmbd* and ZBar with additional image preprocessing as proposed by Gennip et al. [10]. Again, for LETTERNET the percentage of correctly read strings of all three training passes is shown. Column Ushows the results for unprocessed images, Dfor denoise only, UPFS for Uniform PSF and FPSF for Full PSF. A ”-” means no results where given for this setup. Type U D [10] UPSF [10] FPSF [10] LETTERNET LargeNetEmbd*, G19,3,N0.50 1.0 1.0 1.0 1.0, 1.0, 1.0 LargeNetEmbd*, G19,7,N0.50 0 0 0 1.0, 1.0, 1.0 LargeNetEmbd*, G19,11,N0.5- - - - 1.0, 1.0, 1.0 purpose and what influence they have on the appearance of the typeface. In the second part, we want to investigate the practical suitability and answer the question whether this method could in principle replace or complement existing barcode forms in the future. For this purpose, we first compare the robustness of QR codes read by ZBar and LETTERNET against the presented degradations. Our approach is able to adapt to the degradations while ZBar does not exiplcitily handle noise or blur. We are therefore additionally interested in a comparison with an approach dealing with blur and noise in a preprocessing step. Note that the same preprocessing steps could also be used for LETTERNET in addition. All components of this work are artificial, including training data, degradations, etc. In robotics, a typical phenomenon under these circumstances is known as the simto-real gap, which describes the difficulty of applying the approach in reality. We therefore test LETTERNET on real images to investigate the size of the gap and whether realworld applications would be possible. 3.1. Ablation Study In this section variations of LETTERNET as described by Fig. 3are investigated. Two variants are of particular interest here. LargeNetDense is almost equal to LargeNetEmbd with two exceptions: the Embedding layer is replaced by a Dense layer and with one less convolutional block so that the final resolution is only 125×125. MediumNet is almost identical to LargetNetDense without Upsampling layer, but with four times the number of neurons in the Dense layer, so that the resolution remains the same. This way, it can be determined whether more neurons in Dense layers or Upsampling layer lead to improved information transmission. All three network designs are trained three times with various combinations of degradation layer with different parametrization. Tab. 1shows the percentage of recognizable strings from the test dataset, where a string is considered recognizable if each character was correctly reconstructed. Additionally, an example image from one of the three trained networks is depicted. As can be seen, finer structures of LargeNetEmbd are indeed an advantage at higher degradation, but only to a small extent and in some cases, e.g. for degradations B0.15,G15,N0.15,T0.05 it is even a disadvantage. Furthermore, it is evident from LargeNetDense in particular that stronger degradations lead to larger shapes and patterns. This seems logical with stronger blur and noise. 3.2. QR Code Comparison Tab. 1also depicts QR codes for strings from the test dataset. For the string length of this data set, the best error correction type and the smallest QR code type can be applied, i.e. version 1 with ECC level H. To be comparable to LargeNetEmbd, the resolution of the images is 256 ×256, so they have the same number of pixels as the images from LargeNetEmbd. The corresponding degradations of the respective line were applied for the QR code as well. The images of the QR codes in Tab. 1show (in contrast to the other columns) also the degradation to give the reader an impression. Since these are random, however, they are only examples. With a few exceptions, the results clearly show the better recognition rates of all LETTERNET variants presented, even those with only half as many pixels available to transmit the information. 3.3. Comparison with Preprocessing based Approach As LETTERNET is able to adapt to the degradations it is reasonable to compare it with another approach, which also explicitly handles noise and blur before reading the code. The approach by Gennip et al. [10] is not only state-ofthe art, but also well comparable, since it is based on the same degradations and also uses ZBar. It reads the code after noise and blur have been blindly removed based on a regularization method. Tab. 2lists the results of the comparable experiments utilizing LargeNetEmbd*, which is a variant with smaller padding to match the resolution. As can be seen, even with preprocessing LETTERNET’s performance is superior. Note that additional preprocessing steps are also applicable before reading with LETTERNET, which could further improve the results. 3.4. String Lenghts In this experiment, we examine strings longer than 9 characters. Both methods, LETTERNET and QR codes, can encode strings of arbitrary length by simply applying an additional visual representation of the same type. As the previous experiments show, LETTERNET shares properties with huamnoid handwriting. Thus, a natural way to represent longer strings and texts would be to string them together like words. The readability under degradations remains the same. For QR codes, on the other hand, there are different versions for encoding more data, from version 1 to version 40 for up to 1851 characters. Different versions use different numbers of modules per character, and therefore have different susceptibilities to degradation. We expect that higher versions can reduce the overhead of encoding a character including error correction, resulting in a more robust encoding. In this experiment, we encode strings of different lengths up to 1581 characters and evaluate how QR codes of different versions perform under the given degradations Figure 4. Photos of printed LETTERNET codes to be read by the decoder. Figure 5. Photos of LETTERNET codes displayed by a 27” monitor at 4k resolution. compared to LETTERNET. Fig. 6shows the results for selected experiments. The readability under degradations not shown is the same as shown in Tab. 1even for longer strings and is therefore omitted here. It can be seen that different QR code versions perform inconsistently, but cannot be more robust than LETTERNET. 3.5. Sim-to-Real Test This does not yet make any statement about whether LETTERNET can also transmit the data in reality. Training data and also the degradations are artificial and follow a model, which we assume to represent the essential points of reality. In this section, we therefore perform what is known in robotics as a sim-to-real test, where robotic algorithms developed in simulation are tested on real robots. To do this, we selected the best LargeNetEmbd and LargeNetDense for each of the strongest groups of two, three, and four degradations and had images generated for a random string. We printed and displayed these images on paper (Fig. 4) and on a screen (Fig. 5), and then photographed them with an iPhone 11 Mini camera. As can be seen in Fig. 4, the curvature of the paper creates different shadows. Enlarging to DIN A4 also results in typical staircase patterns. Taking a photo from the screen produces a typical Moir´ e pattern. In addition, the backlight of the screen is not optimal and the pictures were not taken exactly perpendicular to the screen. Then the LETTERNET output was cut out by hand (a detector model does not exist at time of writing) and passed this to the decoder part of LETTERNET as a grayscale image. Tab. 3shows that although various effects occur under real conditions that were not modeled (by the degradation layers), the majority were read correctly. Thus, the ability of a CNN to generalize was successful here. The errors suggest that the degradations lead to robust nets that can be applied in reality, but the sim-to-real gap can still be reduced. In the following section, we will develop ideas for doing so. Table 3. Results of the sim-to-real test. The table shows the input string that is expected as output and the actual results when reading the image from a printed paper and from a computer monitor. Type Expected Paper Screen LargeNetDense, B0.5,G60 1HNZ6ZXCG 19NX6290G ✗19NH6Z90G ✗ LargeNetEmbd, B0.5,G60 0AJJJS2QI 0AJJJS2QI ✓0AJJJS2QI ✓ LargeNetDense, B0.4,G40,N0.44YN8CMUAM 4YN8CMUAM ✓4YN8CMUAM ✓ LargeNetEmbd, B0.4,G40,N0.41ILLHSFQ2 1ILLHSFQ2 ✓1ILLHSFQ2 ✓ LargeNetDense, B0.15,G15,N0.15,T0.05 EC2H6N07P EC2H6N07P ✓EC2H6N07P ✓ LargeNetEmbd, B0.15,G15,N0.15,T0.05 1B1C0H22P 1B1C4KH2P ✗1B1C4KH2P ✗ Figure 6. Comparison for different string lengths between LargeNetEmbd and QR codes of different versions, depending on the string length. Results for LETTERNET are extrapolated based on the principle of writing longer strings. 4. Conclusion This paper presents a novel method, called LETTERNET, for transmitting strings consisting of blocks of 9 letters each over a medium such as paper or a display. It is thus comparable to a QR code in an end-to-end learning fashion. We show that it is much more robust to degradations than QR codes, even with a preprocessing filter to remove noise or blur, and can be used in real-world applications. However, more realistic degradations should be applied. Since differentiability is required, a real data set is not directly applicable. However, a generative adversarial network (GAN) that mimics photos from paper, screens, etc. could be trained and then integrated into LETTERNET. References [1] Ismael Benito-Altamirano, David Mart´ ınez-Carpena, Olga Casals, Cristian F´ abrega, Andreas Waag, and Joan Daniel Prades. Back-compatible color qr codes for colorimetric applications. Pattern Recognition, 133:108981, 2023. 3 [2] Tzu-Han Chou, Chuan-Sheng Ho, and Yan-Fu Kuo. Qr code detection using convolutional neural networks. In 2015 International conference on advanced robotics and intelligent systems (ARIS), pages 1–5. IEEE, 2015. 3 [3] Vanita Jain, Yugantar Jain, Hardik Dhingra, Dharmender Saini, MC Taplamacioglu, and MUSTAFA SAKA. A systematic literature review on qr code detection and preprocessing. International Journal on Technical and Physical Problems of Engineering, 13(1), 2021. 3 [4] Yue Jiang, Zhouhui Lian, Yingmin Tang, and Jianguo Xiao. Dcfont: an end-to-end deep chinese font generation system. In SIGGRAPH Asia 2017 Technical Briefs, pages 1–4, 2017. 3 [5] Wendy Cahya Kurniawan, Hiroshi Okumura, Anik Nur Handayani, et al. An improvement on qr code limit angle detection using convolution neural network. In 2019 International Conference on Electrical, Electronics and Information Engineering (ICEEIE), volume 6, pages 234–238. IEEE, 2019. 3 [6] Meng Li, Jian Wang, Yi Yang, Weixing Huang, and Wenjuan Du. Improving gan-based calligraphy character generation using graph matching. In 2019 IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C), pages 291–295. IEEE, 2019. 3 [7] Grace W Lindsay. Convolutional neural networks as a model of the visual system: Past, present, and future. Journal of cognitive neuroscience, 33(10):2017–2031, 2021. 2 [8] Ken Perlin. An image synthesizer. ACM Siggraph Computer Graphics, 19(3):287–296, 1985. 3 [9] Haitao Pu, Mingqu Fan, Jinliang Yang, and Jian Lian. Quick response barcode deblurring via doubly convolutional neural network. Multimedia Tools and Applications, 78(1):897– 912, 2019. 3 [10] Yves Van Gennip, Prashant Athavale, J´ erˆ ome Gilles, and Rustum Choksi. A regularization approach to blind deblurring and denoising of qr barcodes. IEEE Transactions on Image Processing, 24(9):2864–2873, 2015. 3,5,6 [11] S Vijayarani and A Sakila. Performance comparison of ocr tools. International Journal of UbiComp (IJU), 6(3):19–30, 2015. 3 [12] Benzhi Wang, Jingben Xu, Junke Zhang, Guancheng Li, and Xin Wang. Motion deblur of qr code based on generative adversative network. In Proceedings of the 2019 2nd International Conference on Algorithms, Computing and Artificial Intelligence, pages 166–170, 2019. 3 [13] Ratapol Wudhikarn, Phasit Charoenkwan, and Kanokwan Malang. Deep learning in barcode recognition: A systematic literature review. IEEE Access, 2022. 3 [14] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017. cite arxiv:1708.07747Comment: Dataset is freely available at https://github.com/zalandoresearch/fashion-mnist Benchmark is available at http://fashion-mnist.s3-website.eucentral-1.amazonaws.com/. 4 [15] Junhai Zhai, Sufang Zhang, Junfen Chen, and Qiang He. Autoencoder and its various variants. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 415–419, 2018. 3