Digital Twin System of Bridges Group Based on Machine Vision Fusion Monitoring of Bridge Traffic Load

Bridges play an important role in transportation infrastructure systems. Intelligent and digital management of bridges group is an essential part of the future intelligent transportation infrastructure system. This paper proposes a digital twin system for bridges group in the regional transportation infrastructure network, which is interconnected by measured traffic loads. In physical space, a full-bridge traffic load monitoring system based on information fusion of weigh-in-motion (WIM) and multi-source heterogeneous machine vision is set up on the target bridge to measure traffic loads, also lightweight sensors are employed on the bridges group for structural response information. Furthermore, by establishing mechanical analysis models in the corresponding digital space and using the measured traffic loads as links, the working condition perception and safety warning of all bridges in the regional transportation network is achieved, forming an important support for further intelligent transportation infrastructure system. The proposed digital twin system has been preliminarily implemented in a bridges group around Shanghai, China, demonstrating the feasibility of the technical framework proposed in this paper and the bright prospects.


I. INTRODUCTION
W ITH the rapid development of Internet of Things, big data, and cyber physical system, the concept of digitalization of physical world has been proposed gradually and attracted much attention. Among them, the most representative one is the "digital twin" (DT), which was defined as digital representation of a physical entity dynamically through all-round real-time monitoring and control [1]. The concept of DT originated from NASA's Apollo project in 1969 [2]. In 2002, Dr. Michael Grieves also mentioned the concept of DT for the formation of a Product Lifecycle Management(PLM) center, it was simply called "Conceptual Ideal for PLM", which consists of three elements: real space, virtual space, and the link for data flow and information flow [3].
In 2011, the phrase of DT was attached to this concept by Dr. Michael Grieves [4]. In 2012, NASA officially announced the development roadmap of DT, providing a series of new methods, which greatly promoted its development [5]. Since then, DT has been accepted by industry gradually. The concept of DT includes both physical structure in physical space and digital model in cyber space. The most significant thing is to establish an information exchange channel and interaction tasks between the structure and model [6]. In the field of civil engineering, digital models corresponding to physical structure are easily obtained. These digital models include 2D and 3D drawings for design, and mechanical models for structural analysis. These models are mostly used in the design stage of structures, but rarely in subsequent construction and service stages. Although the BIM technology in recent years claims to serve the life-cycle of structure [7], [8], the BIM-based DT for service stage is still in preliminary exploration stage, as there is no information exchange channel established, and also no interaction tasks formed [9]. For example, Chunfeng Wan et al. [10] developed a bridge management system based on the BIM technology. This system has advantages in bridge BIM model fast building and information integration and management, it can improve the management efficiency of bridges. However, for the most important inputs and outputs of the structures, the structural state equation determines the physical mechanical relationship between them. Due to the lack of deep mining of the relationship between input and output information, information exchange and interaction tasks between physical structure and digital model cannot be realized in real. Structural health monitoring system collects data from the sensors which are set up on structure, and then evaluates the health condition and safety risk of the structure [11]. It can not only provide the information exchange channel between structure and model, but also provides various interaction tasks. So the structural health monitoring is the best field to realize DT in civil engineering. In recent years, with the rapid development of wireless sensor network, cloud computing, communication and other basic technologies, the idea and technology of DT has been initially applied in structural health monitoring. Davila et al. [12] proposed an automatic modeling method of BIM model including time-domain data from response sensors to describe the structure and its health monitoring system. This model supports dynamic visualization of key structural performance parameters, allowing seamless data updating and long-term management. On the basis of wireless sensor monitoring network, Bhuiyan et al. [13] designed a cyber-physical system (CPS) of structural event monitoring with WSNs and proposed a novel model-based in-network decision making method, named MODEM. Yuan et al. [14] proposed a CPS-based temporary structure monitoring system to prevent potential structural failures. This system integrated a virtual model of corresponding temporary structure and a physical structure on the construction jobsite. Ozer et al. [15] established a CPS framework including bridge finite element model, smart phone, centralized and distributed computing facilities. Vibration information of structure was obtained by smart phone, and data is transmitted to server wirelessly. Then modal frequency of bridge was identified and finite element model was updated to evaluate the reliability of structure. Kang et al. [16] proposed multimedia knowledge-based bridge health monitoring method using digital twins. In order to synchronize physical and virtual space, data collected by sensors was used to update digital simulation models. Then several extreme situations were simulated to ensure bridge health. However, due to the lack of specific and in-depth understanding of the way information interacts between the physical structure and the digital model. Above work only establishes relationship between them in form, and does not realize the application of DT in civil engineering deeply.
At present, bridge health monitoring system mostly measure and analyze data from the bridge itself [17], [18]. Therefore, information exchange channel and interaction tasks between physical bridge and digital model naturally revolves around the monitored structural responses. Model-based structural health monitoring and diagnosis involves updating the digital model by using the differences between the measured data and the initial digital model information, and then comparing the updated digital model with the initial digital model or the bridge itself to detect damage of the bridge, or to determine and assess its working performance [19], [20]. As the model updating is an ill-posed mechanics inverse problem based on limited measurement points, it is difficult to solve and the result is unsatisfactory [21], this technology is still far from engineering application, and is difficult to be further applied to the technical framework of DT. Another more direct interaction task is, evaluating the status of bridge by comparing measured responses with calculated responses from digital model, then achieve the goal of life prediction, safety early warning and so on [22]. In fact, due to the lack of load information, especially the traffic load information, it is impossible to use digital model to calculate structural responses and make further comparative analysis, and interaction task cannot be established.
In fact, traffic load is the main live load bridge bears. When the traffic load is known, the calculation of corresponding structural responses using digital model becomes a forward problem, which is mature to solve with low difficulty, high efficiency and reliable results. Based on this consideration, measured traffic loads can be used as the information exchange channel between physical bridge and its digital model, and the interaction task can be formed around measured traffic load and calculated responses, which is an ideal way to realize bridge DT. In order to correlate traffic loads to bridge responses, Rui Hou et al. [23] proposed a CPS framework in a highway corridor, to trigger SHM systems to record bridge responses, automatically link the bridge responses with truck weights collected by weigh-in-motion (WIM) stations installed along the corridor. This work shows the possibility of using measured traffic loads as interaction and realizing DT application in the field of health monitoring. The author of this paper has proposed identification methods of full-bridge traffic load distribution based on computer vision in 2019 and 2020. [24], [25], which have preliminarily solved the problem of bridge traffic load measurement. On this basis, antioverturning monitoring method of box girder bridges based on measured traffic load is established [26], [27], as well as simulation method of traffic load model based on machine vision [28]. These studies have shown a good prospect of applying DT to bridges group health monitoring.
For this reason, with the foundation laid by previous research, this paper proposes a bridge DT system connected by measured traffic load, and apply information fusion of machine vision and WIM System to monitor full-bridge traffic loads on the target bridge in a bridges group. Then according to the different purpose of analysis, corresponding finite element models of all bridges in the bridges group are established to run independently on the cloud server. By applying measured traffic load from physical bridge to the corresponding digital model, the model can analyze mechanical responses on-line. And according to the calculation results, evaluation and prediction of the physical bridge working status and safety early warning are realized.

A. Subsystems of Bridge Digital Twin System
As mentioned above, bridge traffic load monitoring is the natural link between bridge in physical space and its DT model in cyber space. In order to realize monitoring and prediction of working status and safety early warning of the bridges group, this paper intends to establish bridges group DT system based on measured traffic loads, and applies the measured traffic loads to the DT models of bridges group to calculate mechanical responses. This system can be divided into three subsystems: traffic load monitoring subsystem based on machine vision and Weigh-In-Motion information fusion, working condition real-time monitoring subsystem of target bridge, and working condition early warning subsystem of adjacent bridges. The specific geographical location of target bridge and adjacent bridge in bridges group are shown in following Fig. 2.
To arrange traffic load monitoring subsystem, a single bridge on main line in the bridges group is selected as monitoring target bridge. Usually, taking the full bridge deck as monitoring range, high-definition cameras and Lidar arrays are arranged longitudinally. Weigh-In-Motion Systems are embedded in pavement at the beginning and end of main bridge on both sides, and ensure that the center line of lateral position of WIM is within the field of first camera's view or within the scanning range of Lidar. The field of view of multiple cameras overlap longitudinally along bridge deck to ensure that full bridge deck is covered with video image signals [24]. Video image obtained by high-definition camera and 3D point clouds scanned by Lidar are used to monitor real-time position of all vehicles passing on this bridge, at the same time Weigh-In-Motion System is used to obtain the vehicle weight and axle load information. Three kinds of information need to be uploaded to cloud server synchronously for centralized processing. The whole layout is shown in following Fig. 3.
As the traffic load is monitored from target bridge, so it can be used on target bridge DT model directly. In fact, a bridge can have any number of digital models. The difference of each digital model lies in the purpose of analysis, method of modeling and precision of model. Therefore, according to the specific calculation purpose, measured traffic loads can be loaded on different digital models, and mechanical responses at specific position on digital model can be calculated, aiming to estimate physical bridge responses at the same position, then structural working condition monitoring and early warning of safety accidents are realized. In order to enhance the monitoring effect, a lightweight response monitoring system can be installed at designated location of physical bridge, then obtain measured responses which corresponding to calculated results. By comparing them, more effective bridge working condition monitoring and safety early warning methodologies can be developed. This is the main function of working condition real-time monitoring subsystem of target Bridge, which shows a complete DT system for target bridge, as information is transmitted two-way between target bridge and its digital model, and information processing process also reflects the interaction between them. The relationship can be shown in Fig. 4.
All bridges in the area share the traffic loads running in transportation network. Therefore, traffic loads on different bridges at different locations should be correlated with each other. The traffic loads monitored on target bridge can moved to other adjacent bridges in bridges group partly. Based on this consideration, DT system for other adjacent bridges in the bridges group can be established. The traffic loads measured on target bridge are still used as the link between other adjacent physical bridges and their corresponding digital models. Moving measured loads to DT models of adjacent bridges, then these bridges' mechanical responses can be calculated. By comparing calculated results with threshold, working status evaluation of these bridges are realized. Since the moving process of measured traffic loads in cyber space takes almost no time, for the DT models of adjacent bridges which located in the forward direction of measured traffic loads, the time to load and calculate can be much less than the time when measured traffic loads are really moved and loaded on physical bridge according to actual speed, so safety early warning with large allowance can be realized. Of course, for the DT models of adjacent bridges located behind the measured traffic loads, also can be loaded in reverse in time, then to replay the past responses history. In addition, if lightweight response monitoring systems are installed on these bridges, comparison between the prediction responses and measured responses can  be realized, which lays a foundation for further interaction task between physical bridges and digital models. The DT system for other adjacent bridges can be shown in Fig. 5.

B. Measured Traffic Load, a Link Between Digital Twin System
Among the three subsystems, the first subsystem is aimed to measure traffic loads, and the second and the third subsystems take the measured traffic loads as a link to realize the interaction between bridge in physical space and digital model in cyber space. It can be seen that measured traffic loads are the key feature of the DT system proposed in this paper. In this system framework, measured traffic loads include following forms: 1) Traffic Load of Single Vehicle (TL 1): Main load parameters of TL 1 include vehicle weight (axle load), speed, real-time position on target bridge (including horizontal and longitudinal position), as well as profile size. When traffic load of this form is moved to digital models of other adjacent bridges, the position information measured on target bridge can be ignored, and the most unfavorable loading position should be considered according to analysis purpose.

2) Traffic Load of Vehicle Queue in Local Single-Lane (TL 2):
In addition to the main parameters of single vehicle traffic load, other parameters like formation shape and group speed of the queue should also be included. When this type of traffic load is loaded on digital model of target bridge, other traffic loads distribution in the rest part of bridge deck should also be considered. When loaded on digital models of other adjacent bridges, the most unfavorable position should be considered. Generally speaking, traffic regulations around the world do not support the formation of heavy vehicle queue in local single-lane, but whether the queue is formed intentionally or accidentally, such loading form is unfavorable to structure and deserve attention.
3) Traffic Load of Full Bridge Deck (TL 3): Weight (vehicle weight and axle load), position, and profile size of all vehicles within full deck area of single bridge (target bridge) need to be monitored precisely. If this measured load is loaded directly on target bridge digital model, it can accurately analyze real-time mechanical responses of target bridge. It can also be loaded on other adjacent bridges digital models after proper cutting longitudinally to predict or replay mechanical responses of adjacent bridge.

4) Statistical Steady State Traffic Load (TL 4):
Usually, it is necessary to continuously monitor full bridge deck traffic load for a long time to get a traffic load model, which includes selection of description parameters, selection of statistical distribution types of parameters, and estimating and updating statistical model of each parameter regularly using measured information. This type of traffic load can be loaded not only on target bridge digital model, but also on digital model of other adjacent bridges after proper adjustment.

5) Statistical Steady State Traffic Load Spectrum (TL 5):
Based on long-term monitoring of the full-bridge traffic load, the statistical values of traffic loads in time domain and frequency domain can be estimated at each loading point on the bridge deck, and can be used to estimate vehicle-induced dynamic response spectrums of structure by loading them on digital model of target bridge and other adjacent bridges.

6) Statistical Cumulative Traffic Load (TL 6):
Based on long-term monitoring of full bridge deck traffic load, the fatigue load at each loading point on bridge deck over a period is calculated, which could be used for fatigue assessment.
These six different forms of traffic loads are obtained by traffic load monitoring subsystem, and are used for interaction between physical bridges and digital models in different scenarios. Among them, the measured single vehicle traffic load can be used for early warning of large transport in transportation network and traffic management, which is of great practical value; the measured local single-lane vehicle queue traffic load can be used to predict responses of some special bridges under least favorable distribution, such as overturning risk prediction of box girder bridges; the measured full bridge deck traffic loads can be used to estimate realtime response of any position in bridge. The above three kinds of traffic load can be directly monitored, while statistical steady-state traffic load, statistical steady-state traffic load spectrum and statistical cumulative traffic load are obtained through statistical analysis on the basis of measured full bridge deck traffic loads, therefore, the last three type can also be called indirect measured traffic loads.
Limited to the length of this paper, in following examples, this paper only illustrates application of the measured full bridge deck traffic load and the measured statistical steadystate traffic load for monitoring and evaluation of bridges group. Before specific application, we need to explain two key technologies in detail. They are full bridge deck traffic load monitoring technology based on machine vision and WIM information fusion, and modeling technology of bridge mechanical analysis DT model for different purposes.

A. Full Bridge Deck Traffic Load Monitoring Technology Based on Information Fusion of Machine Vision and WIM
As mentioned above, the measured bridge traffic loads include six basic forms, among which single vehicle traffic load is the most basic form, and other several forms of traffic load can be derived from single vehicle traffic load. The essence of single vehicle traffic load monitoring is realtime measurement of moving vertical force on bridge deck, which includes size of the force and immediate position of the force. BWIM technology uses bridge itself as a measuring tool to identify size and position of moving force, the principle is using sensors arranged on bridge to measure mechanical responses, and then through mechanical reverse analysis to identify force [29]. Due to the ill-conditioned equations caused by limited measuring points and nonlinear factors of vehiclebridge coupling, the identification quality of this technique for bridge single vehicle traffic load is not ideal, and it is difficult to measure the force in real time and realize full bridge coverage. For the more complex monitoring task of full bridge deck traffic flow loads, this technology is more difficult to be competent. Another idea is to identify separately, the size of force is measured by WIM system [30], and the position of force is monitored by machine vision system based on video monitoring [31]. Full bridge deck traffic loads monitoring technologies proposed by author of this paper in 2019 and 2020 belong to the latter technical route. On the basis of this technology, an improved prototype architecture of full bridge deck traffic load monitoring system based on multisource information fusion is proposed in this paper, as shown in Fig. 6. The proposed prototype system in Fig. 6 consists of hardware subsystem and software subsystem.
Hardware subsystem is the acquisition equipment of vehicle weight, vehicle driving video and vehicle 3D point cloud information. It is composed of multi-source vision sensor network (including high-definition camera and Lidar), WIM system, data field collection and processing system and data communication system. All these devices have mature products in current market, and their technical parameters can meet Processing sequence of multi-source vision information and WIM information in each module is related to the spatial layout of sensor network. For long-span bridges, in order to realize full coverage of visual images on bridge deck, there are many layout forms. One of the most commonly used layouts is that multiple cameras are arranged on deck in turn with the same orientation. Within this arrangement, full bridge deck monitoring area can be divided into three different areas along longitudinal direction of bridge. The first is starting area of bridge, which is limited to WIM sensors layout line (i.e. starting line of first camera's field of view). In this range, SMWI module completes information fusion of WIM and video images. The second is track tracking area on bridge deck, whose range starts from starting line of first camera's field of view to starting line of second visual camera's field of view. In this area, TD, OM and TT modules cooperate with each other to complete location and tracking of all moving vehicles within this range. The third is view handover area. In addition to the TD, OM and TT modules, MFVC module is also needed to complete precise docking of same vehicle in two different fields of view. Entire bridge deck is fully covered by the latter two areas in turn, partially overlapping, as shown in Fig. 7.
In order to realize intelligent perception of full bridge deck traffic loads, the primary task is to identify vehicles from video images or point cloud, which will be completed by Multi-Moving-Target Detection Module (TD). In earlier work, the author of this paper proposed an intelligent target detection scheme for multiple moving vehicle on bridge deck. This scheme designed a YOLO -v3 network structure (as shown in Fig. 8). Using the advantages of this CNN in illumination robustness and recognition accuracy, synchronous high-quality optical detection of overall and rear of vehicle is successfully realized. At the same time, a method of accurately identifying vehicle position and estimating width and length of vehicle is proposed, so as to finally realize real-time accurate identification of traffic flow loads on full bridge deck. Section 4.1 illustrates the recognition quality of this method.
For the vehicles far from the camera, the image may be blurred, in order to further improve the recognition accuracy of real time position, besides the high-definition camera, fixed-base Lidar can also be used at the same time. Using  Lidar to obtain 3D point clouds of bridge deck, and through information fusion, the longitudinal coordinate of vehicle can be corrected, more accurate perception of position can be realized. Dual YOLO-v3 network structure can be used to realize moving vehicle target detection for two different signals, and then feature-level information fusion method can be used to obtain high-precision estimation of vehicle coordinate on bridge deck, as shown in Fig. 9.

B. Modeling Technology of Bridge Digital Twin for Different Mechanical Analysis Purposes
Traffic load is the most important live load of bridge, vehicle-induced response is important for bridge safety and serviceability. The primary task of DT model for target bridge and other adjacent bridges in transportation network is to analyze and calculate mechanical responses under above six measured traffic loads. And the process should be continuous, on-line and real-time. Of course, in addition to vehicle-induced responses calculation, DT models can also be used for other mechanical analysis purposes, such as system identification, performance evaluation, fatigue analysis and so on. According to the different purposes, bridge DT model can be divided into following categories:  Traffic load is a special random load, it is statistically stable when the investigation time is long enough. However, due to the correlation of its temporal and spatial distribution, and the partially nonlinearity of the structure, the independent statistical analysis method at each time can not be used. So, Monte Carlo numerical simulation method should be used on this model usually.

4) Steady-State Dynamic Response Spectrum Analysis DT Model:
Steady-state dynamic response spectrum analysis DT model, is used to analyze various steady-state dynamic response spectrum properties of bridge under steady-state traffic flow loads. Similar to type 2, dynamic effect of traffic loads and road roughness need to be considered in modeling. Also due to the correlation of spatial and temporal distribution, as well as nonlinear influence, response spectrum analysis caused by steady-state traffic loads will be difficult according to classical random structure vibration theory, so numerical simulation is needed with this model.

5) Steady-State Performance Indicator or Damage Indicator Analysis DT Model:
Steady-state performance indicator or damage indicator analysis DT model, is used to analyze various performance indicators or damage indicators of bridge under steady-state traffic flow loads for structural health monitoring and damage diagnosis. These indicators can be defined from the steady-state static responses in type 3, or from steadystate dynamic response spectrums in type 4.

6) Model Updating DT Based on Steady-State Response:
Model updating DT based on steady-state response, is used to update structure model under steady-state traffic loads, and realize perception of structural properties based on monitoring steady-state big data. This requires a lightweight health monitoring system installed on structure to monitor the difference between theoretical and measured steady-state responses in target position, then to continuously update DT model online, so as to realize comprehensive health condition perception of bridge.
7) The Online Fatigue Analysis DT: The online fatigue analysis DT model, is used to analyze fatigue state of structure under measured cumulative traffic loads and to predict fatigue life on-line. In essence, this kind of fatigue analysis is a forward analysis of fatigue stress spectrum or damage cumulative variables under actual traffic loads. Therefore, we do not need to predict hot spots in advance, and can realize fatigue analysis at any position of structure.
According to the purpose and scale of mechanical analysis, different modeling methods of DT model can be selected, range from simple force and displacement method in structural mechanics to multi-scale finite element method. It represents two different modeling style, namely portrait and freehand. Generally speaking, when analyzing entire bridge structure, or evaluating overall performance, we can establish DT model according to the structure mechanics equation sketchily, or choose simple finite element model; when local force analysis is needed, we can choose the more complex beam grid finite element, shell finite element, or even solid finite element. The former has clear physical meanings and are easy to grasp main features, while the latter can obtain more trivial details or precise results. In order to balance the advantages of two modeling styles and obtain higher computational efficiency, bridge digital model can be multi-scale.
There are many foci about bridge in physical space. Under the same traffic loads, there are many overall or local analytical method, and purpose of analysis includes safety, serviceability and durability. So, one bridge can have several different DT models. These DT models are deployed in cloud application server in the form of standalone programs. Each model includes mechanical calculation and analysis module and matched traffic load loader, on standby in cloud. Once a certain form of measured traffic loads loading condition is formed, online mechanical calculation is started.  China. The main bridge is a pre-stressed concrete cable-stayed bridge with single tower and double cable plane, it has unequal span of 105 (m) + 97 (m), and is a tower-pier-girder fixed system. According to the actual size of bridge, finite element model is established by OpenSees. Main girder, pier and tower adopt elastic beam-column element and cable adopts truss element, other mechanical properties and constraints are assigned by corresponding OpenSees modeling commands.
When programing above model, in order to improve the generality of finite element modeling code, structural data is separated from the code and placed in database in a certain format. A script written in Python extract the data needed for structural analysis and generate finite element automatically. The final finite element model program and traffic loads loading program are deployed on cloud server, and related basic data are stored in supporting remote database.

A. Preliminary Realization of Full Bridge Deck Traffic Load Monitoring
According to the prototype framework of DT system for bridges group constructed above, the author of this paper has successively installed some traffic load monitoring system on bridges. Such as Tongji Road T4 Ramp Bridge (Baoshan District of Shanghai, 2016), Yongjiang Bridge (Center of Ningbo, 2017), and Qingfeng Road Ramp Bridge (Center of Ningbo, 2017). Also, several traffic load monitoring schemes with different scales are designed to be built. The specific information of these installed or designed traffic load monitoring systems are given in Table II below. Among them, the traffic load monitoring system of Tongji road T4 ramp bridge in Shanghai was completed in 2016 (shown in Fig. 10). Main purpose of this system is to monitor safety risks of viaducts caused by abnormal traffic loads. This system is equipped with a 4 million-pixel infrared highdefinition camera and a high-precision WIM system, which is in normal operation state and can continuously provide on-site data for scientific research.
The health monitoring system of Yongjiang Bridge includes many kinds of sensors, such as WIM sensor, bridge deck video monitoring facility, accelerometer, strain gauge, GPS, displacement gauge and so on. Traffic load monitoring system of Yongjiang Bridge is formed by reorganization of WIM sensor and bridge deck video monitoring facility. Four Starlight highdefinition cameras constitute a traffic flow video monitoring  II   TRAFFIC LOAD MONITORING SYSTEM INSTALLED OR DESIGNED system covering full bridge deck, two WIM systems are installed at each end of approach bridge to monitor vehicle weight and speed from two directions on bridge, as shown in Fig. 11. Data of this system is transmitted to remote urban bridge monitoring center through public fiber-optic communication network, together with other monitoring data aiming for 24-hour all-weather full bridge deck traffic loads monitoring. So far, the system has been working for nearly three years and has accumulated a large number of traffic video signals and WIM monitoring data.
The traffic load identification technology mentioned in Section 3.1 was used to monitor and identify traffic load of Tongji Road T4 Ramp Bridge and Yongjiang Bridge respectively.
For the traffic load monitoring system of Tongji Road T4 Ramp Bridge, measured WIM data and monitoring video from 3: 00 p.m. to 4: 00 p.m. on January 14th, 2018 are selected, traffic flow loads of full bridge deck with a length of 10 minutes were successfully identified. Considering the demand of traffic loads real time monitoring and the maximum time-consuming of single-step calculation of identification algorithm, video sampling frequency of traffic load is set to 15Hz.
To show the identification effect, traffic loads distribution on full bridge deck at three sampling instants are given in Fig. 12. At each time, position coordinates of each vehicle are accurately measured, the accuracy fully meets the needs of structure analysis of three-span continuous ramp bridge [26]. One rectangle frame represents one vehicle, measured weight values comes directly from WIM system with different color.  It is also convenient to divide one rectangular frame into two or three parts, and each part uses a different color to represent the corresponding axle load value, so that more precise traffic loads can be obtained. These loads can be used to analyze local vehicle load effect combined with local analysis DT model. Two analysis application of DT models using this measured traffic loads will be introduced later in this paper.
For the traffic load monitoring system of YongJiang Bridge, video sensors array covering full bridge deck and two WIM systems on both sides are used. As the single-tower cablestayed bridge is located in city center, and the needs of cityscape, we use the least number of poles on bridge deck to install cameras, so field of view of each camera is designed as shown in Fig. 11. For blind area near the bridge tower, two high-definition cameras covering bridge deck at root of tower are added. For this kind of video sensors array arrangement, we have designed special multi-vehicle track tracking module (TT), multi-field-of-view connection module (MFVC), these works will be shown in other paper due to the paperlength. By selecting monitoring data from 10:00 a.m. to 11:00 a.m. on November 16th, 2020, full bridge deck traffic loads of cable-stayed bridge of 5 minutes was identified, and traffic loads distribution was given in Fig. 13. This measured full bridge deck traffic flows segment will be applied to the DT model of Yongjiang Bridge for real-time load effects analysis.
Statistical steady-state traffic flow loads of existing roads and bridges are actually a model describing a certain cross section of road or bridge deck. In order to obtain measured statistical steady-state traffic flows of Yongjiang Bridge, the cross section locating in bridge tower is taken as representative. The process includes, firstly, using a video camera located in middle of tower to obtain video image of each vehicle passing through this cross section, then using a special images target detection algorithm to detect vehicle profile and wheels, at the same time, using a special optical measurement algorithm to realize real-time on-line recognition of vehicle length, front and rear suspension length and lateral position. Secondly, combined with vehicle weight monitoring value from WIM system, measured traffic loads in local area near the bridge tower can be obtained. If this kind of monitoring and identification lasts for a longer period, observed parameters will gradually into stable state. At this time, statistical characteristics of each parameters can be obtained by statistical analysis, and then measured statistical steady state traffic flows can be obtained. Also, with the identification of above key parameters, we can measure a more precise statistical steady state traffic flow loads, and its core feature is a microscopic traffic flow model based on Intelligent Driver Model (IDM), which makes it closer to actual traffic loads. Fig. 14 shows a segment of this measured traffic flow loads. It includes four vehicles, and the speed, weight, wheel and center position coordinates of each vehicle are gene by the measured information.
Long-term static and dynamic responses of target bridge and adjacent bridges can be evaluated by applying this measured statistical steady-state traffic flow loads to corresponding DT model. The specific application will be given below.

B. Preliminary Realization of Bridge Digital Twin System Based on Measured Traffic Load
Traffic load is the most important live load for bridge, it is not only main operating load of bridge, but also an important factor affecting bridge safety, serviceability and durability. In fact, design and recheck of bridge during design stage and status evaluation during service stage are all based on traffic load effect. In design stage, design traffic load is based on empirical investigation, and is generally regulated as lane load and vehicle load, which are quite different from real traffic load during service period. This design load can only be used for analysis and recheck in design stage. In service stage, traffic load loaded on bridge is usually less than the design level, but in some cases, it may exceed the level, which may lead to bridge safety accidents. In addition, some special service stage analysis, such as working status prediction, safety early warning, fatigue assessment, and long-term performance evaluation, all need several measured traffic loads described above.
The system proposed above has realized the monitoring of actual traffic loads during service stage of bridge. Next, these measured traffic loads can be loaded on bridge DT models for different analysis purposes. In this paper, three analysis examples of bridge DT models under measured traffic loads are given in order to preliminarily illustrate the usage and effectiveness of the system proposed in this paper.
1) Case1. Measured Traffic Load and Its Short-Term Effect: Overturning Risk Assessment: The main purpose of traffic load monitoring of Tongji Road T4 Ramp Bridge is to realize real-time monitoring of overloaded vehicles and abnormal queue formed by them. According to the identification results, some possible bridge safety accidents, like overturning and fracture, are predicted, and then to control the traffic of urban viaduct. Due to limitation of urban space, many viaducts and ramp bridges are single-column-pier box girder bridges. The anti-overturning capacity of these bridges is very limited, so overturning accidents may occur under some abnormal traffic loads.
In order to predict overturning risk of single-column-pier box girder bridges under measured traffic loads, we establish a DT model for real-time overturning early warning, shown in Table I. A segment of full bridge deck traffic flow from Tongji Road T4 Ramp Bridge is loaded on this model, aiming at calculating support reaction of middle single pier, and risk of overturning can be judged by whether the reaction force is zero. The analysis detail can refer to previous work [23]. Here, only the results are given, as shown in Fig. 15. Fig. 15 shows the support reactions calculated by realtime overturning early warning DT model of box girder bridges under measured traffic flow loads (excluding the dead load). Support2 represents the middle bearing of the bridge, and Support 1-1, Support 1-2, Support 3-1 and Support 3-2 represent the four bearings on the side pier. The figure shows that the middle bearing is always under pressure, while the other four bearings are in an alternating state between positive and negative. In the whole process, the minimum value of the bearing reaction force is about-5kN, while the calculated side bearings reaction force is about 787kN under the dead load. So the bearing reaction force caused by vehicle load are negligible, and there is no risk of overturning during the monitoring process in this bridge. This real-time overturning early warning DT model adopts a simple structural mechanics principle, the analysis speed is fast, the result is accurate, and the process is simple and feasible. In addition, this measured traffic loads segment from target bridge can be used in other box girder bridges on the same route before traffic loads actual arrived, then to predict overturning risk. It can be seen that, interaction task between digital models in cyber space and box girder bridges in physical space is realized to ensure safety of bridges and traffic.
2) Case 2. Measured Traffic Load and Its Short-Term Effect: Prediction of Transient Vehicle-Induced Response of Cable-Stayed Bridge: It is interesting to use the DT model  to calculate bridge transient responses under measured traffic loads. For two reasons, first, for the target bridge, it can break through the limit of number of measuring points and realize the real-time calculation of any mechanical responses at any position of bridge, which is equivalent to place sensors at any position. Also, for the key position where sensor has been arranged, the current working statue of structure can be judged by comparing measured values with semi-measured and semi-theoretical values calculated by DT model. Second, for other adjacent bridges, measured traffic load can be moved and loaded into the corresponding digital model in the most unfavorable way to realize predict or playback of accident.
To illustrate this function, measured traffic flow loads segments obtained from Tongji Road T4 Ramp Bridge and Yongjiang Bridge in section 4.1 are respectively loaded on the DT model of vehicle-induced response analysis of Yongjiang cable-stayed bridge. Loads segment 1 is loaded with unfavorable position, loads segment 2 is loaded with real measured position. Fig. 16 shows the change of longitudinal displacement of top of tower, deflection of mid-span in west side and tension of SW.1 cable of Yongjiang cable-stayed bridge in these two cases. Fig. 16 (a), (b), (c) show the responses of cable-stayed bridge under the most unfavorable loading with measured traffic loads segment 1 from Tongji Road T4 Ramp Bridge, and Fig. 16 (d), (e), (f) show the responses of cable-stayed bridge under its own measured traffic loads segment 2. No matter which loads segment is used, the responses calculated are very consistent with immediate distribution of vehicle on bridge deck according to frame-by-frame comparison, result accuracy is related to precision of DT model. In calculation process of vehicle-induced responses, single-step calculation time is within 0.05s, which is less than traffic load video sampling interval. It can be seen that within allowable accuracy range, the relatively simple DT model can reduce time-consuming of analysis and calculation, thus making real-time on-line structural responses analysis, as well as other further online evaluation and safety early warning possible.

3) Case 3. Measured Statistical Steady-State Traffic Load and Its Long-Term Effect Evaluation: Statistical Analysis of
Cable-Stayed Bridge Response: The advantage of establishing a real-time monitoring system of bridge traffic load is not only to monitor short-time traffic flow loads of full bridge deck, but also to obtain a large number of traffic loads data by longterm and equal time interval sampling. Based on statistical analysis and modeling of these large quantities of measured data, several type of statistical steady-state traffic loads models can be obtained. These statistical steady-state measured traffic loads, can be loaded on DT model of target bridge and other adjacent bridges, and to find statistical regularities of longterm static responses and dynamic response spectrums. It can also be used to evaluate and predict fatigue life.
To illustrate, the statistical steady-state traffic flow loads of Yongjiang Bridge were measured using above technique and loaded on the DT model of response analysis of cable-stayed bridge. Because the random loads loaded on different joints of main girder are correlated, this kind of responses analysis   distribution of full bridge deck at a specific time is generated according to the Intelligent Driver Model (IDM); thirdly, apply the loads distribution into DT model to analyze and calculate structural responses. Repeat these steps until enough vehicleinduced response samples are obtained, and then analyze statistical regularities of responses in any desired positions. Some analysis results are shown in Fig. 17 and 18 below. Fig. 17 shows the semi-measured and semi-theoretical responses analysis results of DT model of Yongjiang Bridge. It can be seen that, under the statistical steady-state traffic flow load, whether it is the strain at bottom of mid-span girder, the tension of SW.2 cable, or the displacement at top of tower, when the samples are large enough, theirs values are all statistically stable. Among them, the probability density distribution of strain at bottom of mid-span girder and tension of SW.2 cable are positive-skewed distributions, the mean value of strain and cable tension is larger than the mode. For the strain, it basically increases under traffic loads, and for the cable tension, in most cases will increase, in a small number of cases will decrease, depending on the influence line function of this cable tension. The probability density distribution for longitudinal displacement of top of tower is normal distribution, and is symmetrical under traffic loads, indicating the structure is nearly longitudinal symmetrical. These results show that responses of cable-stayed bridge has rich statistical meaning, and reflects regularities of structure under big data loading, which is worthy of further study. Fig.18 shows the spatial probability density distribution curve of deflection at bottom of main girder. On the whole, the spatial distribution surface composed of probability density curve of deflection (blue line) is a curved ridge surface, and the projection of the line connecting the highest point of each probability density curve represents initial configuration of main girder (without traffic loads). From the local perspective, the width of each probability density distribution curve indicates distribution range of deflection value at this point, so the connection line of the starting (and ending) points of each probability density distribution curve represents deflection envelope of main girder in simulated case. These contents indicate, the probability density spatial distribution curve contains rich structural mechanical characteristics, and can be used to excavate structural statistically behaviors regularities under big data loading.
The above two figures show that, bridge DT model can realize structural long-term effect evaluation and statistical regularities study under measured statistical steady-state traffic load. How to use the result of evaluation to deduce health status of bridge prototype in physical space and form more interaction tasks between two spaces will be the future exploration.

V. CONCLUSION
Traffic load is the main live load bridges bears and affects function and reliability of bridge in service period. Based on machine vision fusion monitoring of bridge traffic loads, this paper constructs a digital twin system for regional bridges group, and realizes the interaction between physical bridge and digital model. By monitoring the traffic loads in physical bridge, and loading the actual traffic loads to the corresponding digital twin model, semi-measured semi-theoretical responses are obtained. By comparing it with measured responses from physical bridge, safety warning and working status evaluation of physical bridges can be realized.
The technical framework of traffic load monitoring proposed in this paper has been realized in author's previous research [24], [25]. The measured traffic loads of different forms from two bridges meet the real-time requirements of the actual monitoring system, and can be applied to various digital twin models. The proposed classification method of measured traffic loads and bridge digital twin model, as well as the modeling ideas are all proved reasonably through real case study. All these show that using measured traffic loads to realize interaction between physical bridge and digital model is the most direct and reasonable way. It can realize the interconnection and collaboration of transportation infrastructures, so as to promote the construction of future intelligent transportation infrastructure systems.