During the Cross-CPP project, Meteologix has built several prototypes for CPP based services that are designed to make use of CPP data from “foreign” sensors to fully exploit the possibilities of big data for meteorological applications.
For that, Meteologix has taken various perspectives on the provided data and envisioned several exploiting strategies for varying meteorological use cases. The main goal of Meteologix’ participation was to build a data environment within our meteorological infrastructure to assimilate sensor data from CPPs most efficiently as ordinary assimilation procedures expect high-quality measurement data from weather stations. However, the data derived from CPPs has several features that are different from classical observational data used in weather data processing contexts. For example, CPP data is potentially much denser (in regard to both time and location) and often less accurate as more measurement errors are to be expected due to e.g., car color, driving speed, placement of the sensor etc. Hence, it was necessary to conceptualize and subsequently develop innovative plausibility checks that validate the incoming CPP data, and also to develop a weather model that was able to handle the density and small-scale measurement differences in a way that would not lead to the rejection of most incoming signals.
UltraHD city model prototype
Meteologix has thus made great effort to build a full-physics extremely small-scale weather model prototype - called the UltraHD city model - that is able to resolve weather dynamics and improve the parameterisation of skin temperature and applied fluxes on a 100m grid. This resolution together with a newly invented plausibility check algorithm for CPP data allows efficient and enhanced data assimilation of CPP data, e.g., from cars or buildings or moving sensors like bicycle temperature sensors. The UltraHD is nested into the Meteologix’ 1x1km SuperHD, which was thus also enhanced for the purpose of this project as the UltraHD is dependent on high-quality boundary data as starting input for its own calculations.
To clarify, the UltraHD is not a classic downscaling model, but its own full-physics model that explicitly resolves larger turbulent motions in the atmospheric boundary layer (so called Eddies). These large Eddies are crucial to describe mass and energy transport in the boundary layer. Also, microphysical processes which describe phase transitions in the atmosphere, e.g., cloud/rain formation, freezing and sedimentation of snow and rain, are treated explicitly at the 100m scale by the two-moment microphysical parameterisation.
This UltraHD prototype currently provides output for the most important weather parameters (e.g., 2m temperature, 2m dew point, total column condensate, 10m wind, surface pressure, precipitation rain/snow), which enables it to assimilate sensor data for these parameters into the UltraHD to enhance its initializing fields and subsequently its predictions. Those predictions can be visualized for example as forecasting maps.
Figure 1: UltraHD model map of 2m temperature Interlaken (CH)
The difference in resolution and accuracy can be best demonstrated by model output comparison. Figure 2 shows the same map excerpt for the same time by two leading forecasting models (GFS and ECMWF), the third picture shows the corresponding model output of the UltraHD.
Here are some model outputs and UltraHD simulations:
Figure 2: 2m temperature region Interlaken (CH), 17.2.21, 00z): GFS output (left), ECMWF output (middle), UltraHD output (right)
Figure 3: UltraHD model output: 2D RGB Clouds/Rain Convection in Mountains
Figure 4: UltraHD 3D simulated roundshot Lauberhorn/ fair weather convection
Beginnings of small-scale air quality forecasting:
The UltraHD model was also extended with a simple atmospheric chemistry model to model air quality including the following reactants: nitrous oxide, nitrous dioxide, nitric acid, ozone, hydroxyl radical, hydrogen peroxide, volatile organic compounds and organic peroxyradicals. Furthermore, a set of 22 additional equations was implemented including radiation dependent photolysis of ozone, reactions with atmospheric humidity and the formation of nitric acid. In addition, equations for the emission and deposition of passive atmospheric tracers like pm10 and pm2.5 were included.
Although the additional variables and equations increased the memory and computational demand of the model significantly, it was possible to build a three-dimensional setup to simulate the lower troposphere for the region Tübingen/Stuttgart (GER) in real-time. A detailed analysis of infrastructure was necessary to parameterize emission of the NOx and PM compounds. This was done using OpenStreetMap data and identifying highways and certain urban structures as possible sources.
For validation of the UltraHD, we have nested it in several testing areas: Tübingen/Stuttgart (GER), Greater Basel (CH), Wolfsburg (GER), Berlin (GER), Leipzig (GER), and areas in the Swiss mountains: region around Interlaken (CH). A quantitative validation of the UltraHD is still ongoing as well as its further development.
Smart building ML-based weather forecasts
Besides the invention of a small-scale weather model for CPP data ingestion, we wanted to test the usage of CPP data for other meteorological applications that are more industrial user focussed, like the enhancement of already existing meteorological CPP data outputs of building or industrial sites. For this, we co-operated with consortium partner Siemens, accessed their building data via the Cross-CPP marketplace, and helped them to refine the weather predictions they received for their smart buildings by running machine learning models based on the historical data of their on-site weather stations, that are usually located at the roof of the building. These models learned by the historical measurement data and historical weather model output the building specific deviations and biases, and subsequently reduced them for new forecasts.
By applying these models, we were able to reduce the forecast deviations for the Siemens buildings by ~40%, resulting in a remaining error of just ~1.49 °C between the so enhanced forecast for the building and the actual observation.
Figure 5: Exemplary time series of the observed air temperature (purple), air temperature from DMO (green) and the predicted air temperature (blue) for one week in March.
Virtual Radar
Another approach for using CPP data for meteorological purposes was to explore the possibilities of utilizing car rain sensor data and car wiper data. We conceptualized a “Virtual Radar” based on CPP sensor data instead of actual weather radar data that could be used to gain insights into real time rain events in regions that do not have expensive weather radars or where those are not publicly available.
We investigated which development steps would be necessary to build a service like that, and found that the output and measurement of car rain sensor data and wiper data is not standardized across all vehicles and vehicle types, thus effort is needed to account for those differences. Furthermore, the driving speed and the angle of the windshield needs to be considered when deriving precipitation data from windshield sensors as they play a crucial role on how much rain actually lands on the windshield.
Afterwards, the measured rain signals need to be calibrated regularly by using standard rain measurements (like weather radars) to gauge the output properly. And finally, the so collected precipitation data needs to be aggregated and visualized in almost real-time precipitation maps, which could then potentially used to issue warnings (e.g., flash flood warnings).
Our project results indicate that the car sensors (at least in the VW cars in this project) are very sensitive even to small droplets. The maximum value of 10 is reached in a lot of weak rain events (low radar reflectivity <10 dBZ), suggesting that the sensors are triggered not (only) by intensity but instead (also) by droplet number. Rainfall intensity measured by traditional weather stations (in liter/m² water equivalent) is a function of drop size and number.
Despite the preliminary findings regarding the limitation of the intensity range, the car data sensors show potential in some applications. For example, the discrimination between radar echoes reaching the ground and those evaporating beforehand. Or for filling the remote sensing gap of drizzle, i.e., very fine/small droplets which are rarely detected by typical radar devices. On the other hand, cars will likely not be able to replace a precipitation radar, at least not in the given configuration with a capped intensity value, because summer-time intensities are expected to be of intensity 10 (highest intensity in the testing vehicles) almost always.
During the Cross-CPP project, we investigated the development steps to create a CPP-based virtual radar and also developed the calibration procedure, and processing algorithms to extract a valid and reliable signal from some vehicle types, however, intense testing with more car rain sensor data, and subsequently further calibration periods are needed to fully develop a fully functional independent prototype service based on these theoretical proof-of-concept procedures. Nevertheless, based on our tests during Cross-CPP, Meteologix is confident that a service like that is possible and a valuable addition to conventional weather observation sources.
Figure 6: Scatter plot of cars’ rain intensity sensor (y-axis) versus precipitation radar reflectivity (x). Red data points have been detected as implausible, black points have not been used during calibration because of low confidence in low-dBZ radar values.
Storm-cell intersection program
Lastly, Meteologix participated in the joint service project of a weather-based navigation for cars that warns the driver in advance of and during their trip of significant weather events. For this, we developed a real-time storm cell tracking to detect thunderstorms and track their pace and direction. This storm-cell tracking enables us to intersect the driver’s route with the forecasted thunderstorm’s path to issue warnings to the driver. Get more information on this service at the service description from VW.
In sum, Meteologix has come to the conclusion that CPP data can aid the expansion of meteorological solutions to new market sectors like smart technology as well as logistics and more user centred services despite the classical weather forecasting app. Meteologix has also shown that the quality of CPP data is diverse, but usually sufficient for meteorological purposes as its data quality deficiencies in comparison to classical meteorological observations can potentially be compensated by data density as CPP data usually is available in much larger amounts than professional measurements.
Meteologix has created a demonstrator video that shows you the rationale behind each prototype and its business use case. You can find the video with the application scenarios for the presented developments here:
Meteologix' service descriptions:
That's all from the Meteologix Team! We hope you enjoyed these insights into our inventions, we surely enjoyed working on them :)
Let us know if these innovations have sparked your interest and write us an email at info@kachelmann.com