Abstract: This paper presents a piecewise nonlinear approximation algorithm for computing transcendental functions, with a focus on the Sigmoid function, which is widely used in neural networks. A segmentation method is proposed that leverages the symmetry of the Sigmoid function and its non-uniform derivative behavior. The impact of the segmentation approach and the order of the approximating polynomial on the accuracy of the results is analyzed. The algorithm is implemented on an FPGA, using a third-order polynomial to process the Sigmoid function. The implementation achieves a precision of up to 10-5 and a maximum frequency of 127.327 MHz, meeting the demands of high-speed and high-precision computation.
In real-time image processing, digital signal processing, and other fields, high-speed computation of nonlinear functions is often required [1]. In artificial neural networks, a large number of nonlinear functions must be computed, making the efficient processing of these functions crucial. Among them, the Sigmoid function is one of the most commonly used. Current research on the implementation of the Sigmoid function can be broadly divided into software and hardware approaches. However, due to the slower execution speed and lower parallelism of software, it cannot meet the requirements for fast processing [2]. Therefore, with the rapid development of very large-scale integrated circuits, it has become increasingly important to study how to implement Sigmoid function processing using hardware.
FPGAs, with their reconfigurable nature, offer a flexible and powerful solution for high-speed computation involving nonlinear functions like the Sigmoid. Common methods for implementing the Sigmoid function on FPGAs include lookup tables, CORDIC algorithms, Taylor series expansions, and piecewise linear approximations. While lookup tables are easy to implement, they require significant memory resources as accuracy increases. The CORDIC algorithm, although effective, becomes slower with higher precision due to increased iterations. Taylor series expansion also consumes a lot of resources. Piecewise linear approximation, while faster than many alternatives, still struggles with achieving high accuracy due to the use of low-order polynomials over small intervals. Thus, there is a need for a more accurate and efficient approach.
To address these challenges, this paper employs a traditional piecewise nonlinear approximation method tailored for the Sigmoid function. Although similar techniques have been applied to hyperbolic tangent functions in previous studies [8], no clear basis for segmentation or consistent accuracy across segments was provided. In response, this paper proposes a segmentation strategy based on the symmetry and derivative characteristics of the Sigmoid function. Using the least squares method from numerical analysis, a reasonable segmentation is determined, and the accuracy differences under uniform processing are analyzed. A hardware structure is designed using a hardware description language, and the design is verified on a Xilinx Virtex-5 XC5VLX110T device. The results are evaluated in terms of resource usage, computation speed, and accuracy, demonstrating the effectiveness of the proposed method.
1. Piecewise Nonlinear Fitting Scheme and Result Analysis of the Sigmoid FunctionThe basic principle of piecewise nonlinear approximation involves dividing the function into segments and fitting each segment with a high-order polynomial. This transforms the complex nonlinear function into a set of polynomial calculations, which are computationally simpler and faster to execute.
According to the Taylor series expansion, the function can be approximated around a given point. As more terms are included in the expansion, the approximation error decreases. However, the value of the (N+1)th derivative directly influences the error introduced by the Nth-order polynomial. If the absolute value of the (N+1)th derivative is large at a particular point, the approximation error will be larger, requiring a smaller interval for that segment. Conversely, if the derivative is small, a larger interval can be used. By dynamically adjusting the segmentation intervals based on the derivative values, the overall accuracy of the approximation can be improved.
This approach is applied to the Sigmoid function, whose properties and derivatives are analyzed in detail. As shown in Figure 1, the Sigmoid function F(x) is symmetric about the point (0, 0.5), meaning that only half of the domain needs to be computed, and the other half can be derived using the symmetry property. Additionally, the fourth derivative of the Sigmoid function reaches a maximum near x = 1 and diminishes as x moves away from this point, eventually approaching zero. Based on this observation, a third-order polynomial is used for approximation, as it provides sufficient accuracy when the segmentation is appropriately adjusted according to the derivative behavior.
(1) The Sigmoid function is symmetric around (0, 0.5). This allows for computational efficiency by calculating only one half of the domain and deriving the other using symmetry.
(2) On the positive side, the fourth derivative peaks near x = 1 and decreases on both sides, indicating that the optimal segmentation should be narrower around this region to maintain high accuracy.
Air Bar Disposable Pod Device
The SUORIN Air Bar is a Disposable Device filled with 1.8ml of salt nic e-liquid and comes packaged as a disposable unit. Allowing the user to enjoy a delicious vape with zero up keep, and is a simple trip to the trash once the device has been used up. Each Air Bar Disposable Device features an estimated 500 puffs, and has an internal 380mAh battery which is to be disposed of once the life cycle ends. The Air Bar Disposable accommodates 1.8ml of Salt Nicotine E-Liquid with a nicotine level of 50mg.
Features:
500 Puffs Approx.
Draw-Activated Firing Mechanism
Integrated 380mAh Battery
1.8mL Juice
Disposable - Non-Rechargeable & Non-Refillable
Manufactured by Suorin
Shake Shake Airbar,Diamond Air Bars,Air Bars Disposable,Peach Air Bar
Shenzhen Ousida Technology Co., Ltd , https://en.osdvape.com