Svoboda | Graniru | BBC Russia | Golosameriki | Facebook

Meta-Learning-Based Spatial-Temporal Adaption for Coldstart Air Pollution Prediction

Authors:

Lin Zhang Academic Editor:

Alexander HošovskýAuthors Info & Claims

International Journal of Intelligent Systems, Volume 2023

https://doi.org/10.1155/2023/3734557

Published: 01 January 2023 Publication History

Abstract

Air pollution is a significant public concern worldwide, and accurate data-driven air pollution prediction is crucial for developing alerting systems and making urban decisions. As more and more cities establish their monitoring networks, there is a pressing need for coldstart model training with limited data accumulation in new cities. However, traditional spatial-temporal modeling and transfer learning schemes have been challenged under this scenario because of insufficient usage of available source data and suboptimal transferring strategy. To address these issues, we propose a meta-learning-based spatial-temporal adaptation solution for coldstart air pollution prediction. Our approach is a model-agnostic framework that enables a given backbone predictor with adaption ability across different space and time locations. Specifically, it learns a factorization of the available source data distribution and recognizes the target city as one of its components, greatly reducing the data accumulation requirement and providing coldstart capability. Furthermore, we design a novel bidirectional meta-learner that can simultaneously leverage task embeddings learned from data and features constructed based on prior knowledge. We conduct comprehensive experiments on both synthetic and real-world air pollution datasets of four distinct pollutants. The results demonstrate that our proposed method achieves a 5.2% lower 24-hour prediction mean absolute error (MAE) than pretraining and fine-tuning solutions when facing a new city with only 200 hours of data, which empirically verifies the effectiveness of our approach as a coldstart training solution.

References

[1]

W. H. Organisation, Ambient Air Pollution: A Global Assessment of Exposure and burden of Disease, World Health Organization, Geneva, Switzerland, 2016.

[2]

A. Masood and K. Ahmad, “A review on emerging artificial intelligence (ai) techniques for air pollution forecasting: fundamentals, application and performance,” Journal of Cleaner Production, vol. 322, 2021.

[3]

Z. Luo, J. Huang, K. Hu, X. Li, and P. Zhang, “Accuair: winning solution to air quality prediction for kdd cup 2018,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1842–1850, Anchorage, AK, USA, June 2019.

[4]

B. Zhang, Y. Rong, R. Yong, D. Qin, M. Li, G. Zou, and J. Pan, “Deep learning for air pollutant concentration prediction: a review,” Atmospheric Environment, vol. 45, 2022.

[5]

M. Wang and W. Deng, “Deep visual domain adaptation: a survey,” Neurocomputing, vol. 312, pp. 135–153, 2018.

Digital Library

[6]

M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars, “A continual learning survey: defying forgetting in classification tasks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3366–3385, 2022.

[7]

J. Wang, C. Lan, C. Liu, Y. Ouyang, T. Qin, W. Lu, Y. Chen, W. Zeng, and P. Yu, “Generalizing to Unseen Domains: A Survey on Domain Generalization,” IEEE Transactions on Knowledge and Data Engineering, vol. 45, 2022.

[8]

J. Ma, Z. Li, J. C. Cheng, Y. Ding, C. Lin, and Z. Xu, “Air quality prediction at new stations using spatially transferred bi-directional long short-term memory network,” The Science of the Total Environment, vol. 705, 2020.

[9]

I. H. Fong, T. Li, S. Fong, R. K. Wong, and A. J. Tallon-Ballesteros, “Predicting concentration levels of air pollutants by transfer learning and recurrent neural network,” Knowledge-Based Systems, vol. 192, 2020.

[10]

Y. Zhang, Q. Lv, D. Gao, S. Shen, R. P. Dick, M. Hannigan, and Q. Liu, “Multi-group encoder-decoder networks to fuse heterogeneous data for next-day air quality prediction,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}, pp. 4341–4347, Macao, China, July 2019.

[11]

H. Yao, Y. Liu, Y. Wei, X. Tang, and Z. Li, “Learning from multiple cities: a meta-learning approach for spatial-temporal prediction,” in Proceedings of the The World Wide Web Conference, pp. 2181–2191, New York, NY, USA, August 2019.

[12]

Y. Zhang, Y. Li, X. Zhou, J. Luo, and Z.-L. Zhang, “Urban traffic dynamics prediction—a continuous spatial-temporal meta-learning approach,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 13, no. 2, pp. 1–19, 2022.

Digital Library

[13]

Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a few examples: a survey on few-shot learning,” ACM Computing Surveys, vol. 53, no. 3, pp. 1–34, 2020.

Digital Library

[14]

Y. Du, J. Wang, W. Feng, S. Pan, T. Qin, R. Xu, and C. Wang, “Adarnn: adaptive learning and forecasting of time series,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 402–411, Queensland, Australia, November 2021.

[15]

J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” 2018, https://arxiv.org/abs/1801.06146.

[16]

T. M. Hospedales, A. Antoniou, P. Micaelli, and A. J. Storkey, “Meta-learning in neural networks: a survey,” ieee transactions on pattern analysis and machine intelligence,” 2021, https://arxiv.org/abs/2004.05439.

[17]

M. Garnelo, D. Rosenbaum, C. Maddison, T. Ramalho, D. Saxton, M. Shanahan, Y. W. Teh, D. Rezende, and S. A. Eslami, “Conditional neural processes,” in Proceedings of the International Conference on Machine Learning, pp. 1704–1713, PMLR, Baltimore, MA, USA, June 2018.

[18]

C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the 34th International Conference On Machine Learning, Ser. Proceedings of Machine Learning Research, D. Precup and Y. W. Teh, Eds., pp. 1126–1135, Baltimore, MA, USA, August 2017.

[19]

T.-D. Hoang, N. M. Ky, N. T. N. Thuong, H. Q. Nhan, and N. V. C. Ngan, “Artificial intelligence in pollution control and management: status and future prospects,” Artificial Intelligence and Environmental Sustainability, vol. 34, pp. 23–43, 2022.

[20]

Z. Wu, Y. Wang, and L. Zhang, “Msstn: multi-scale spatial temporal network for air pollution prediction,” in Proceedings of the IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA, June 2019.

[21]

Y. Zheng, X. Yi, M. Li, R. Li, Z. Shan, E. Chang, and T. Li, “Forecasting fine-grained air quality based on big data,” in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2267–2276, Singapore, December 2015.

[22]

P. Zhao and K. Zettsu, “Mastgn: multi-attention spatio-temporal graph networks for air pollution prediction,” in Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), pp. 1442–1448, Manhattan, New York, USA, July 2020.

[23]

G. T. Wilson, “Time series analysis: forecasting and control,” in Journal of Time Series Analysis, john wiley and sons inc, Hoboken, NY, USA, 5 edition, 2016.

[24]

L. Zhang, J. Lin, R. Qiu, X. Hu, H. Zhang, Q. Chen, H. Tan, D. Lin, and J. Wang, “Trend analysis and forecast of pm2. 5 in fuzhou, China using the arima model,” Ecological Indicators, vol. 95, pp. 702–710, 2018.

[25]

M. Dong, D. Yang, Y. Kuang, D. He, S. Erdal, and D. Kenski, “Pm2. 5 concentration prediction using hidden semi-markov model-based times series data mining,” Expert Systems with Applications, vol. 36, no. 5, pp. 9046–9055, 2009.

Digital Library

[26]

I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” Advances in Neural Information Processing Systems, vol. 27, pp. 3104–3112, 2014.

Digital Library

[27]

A. Tiwari, R. Gupta, and R. Chandra, “Delhi Air Quality Prediction Using Lstm Deep Learning Models with a Focus on Covid-19 Lockdown,” 2021, https://arxiv.org/abs/2102.10551.

[28]

Y. Liang, S. Ke, J. Zhang, X. Yi, and Y. Zheng, “Geoman: multi-level attention networks for geo-sensory time series prediction,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pp. 3428–3434, Stockholm, Sweden, June 2018.

[29]

J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin, “Convolutional sequence to sequence learning,” in Proceedings of the International Conference on Machine Learning, pp. 1243–1252, Baltimore, MA, USA, June 2017.

[30]

H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: beyond efficient transformer for long sequence time-series forecasting,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 12, pp. 11106–11115, 2021.

[31]

B. Liu, X. Yu, J. Chen, and Q. Wang, “Air pollution concentration forecasting based on wavelet transform and combined weighting forecasting model,” Atmospheric Pollution Research, vol. 12, no. 8, 2021.

[32]

X. Yi, J. Zhang, Z. Wang, T. Li, and Y. Zheng, “Deep distributed fusion network for air quality prediction,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 965–973, New York, NY, USA, August 2018.

[33]

J. Zhang, Y. Zheng, and D. Qi, “Deep spatio-temporal residual networks for citywide crowd flows prediction,” in Proceedings of the Thirty-first AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, August 2018.

[34]

A. Alléon, G. Jauvion, B. Quennehen, and D. Lissmyr, “Plumenet: Large-Scale Air Quality Forecasting Using a Convolutional Lstm Network,” 2020, https://arxiv.org/abs/2006.09204.

[35]

F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009.

Digital Library

[36]

J. Toutouh, “Conditional generative adversarial networks to model urban outdoor air pollution,” in Proceedings of the Ibero-American Congress of Smart Cities, pp. 90–105, Berlin, Germany, May 2020.

[37]

Y. Lin, N. Mago, Y. Gao, Y. Li, Y.-Y. Chiang, C. Shahabi, and J. L. Ambite, “Exploiting spatiotemporal patterns for accurate air quality forecasting using deep learning,” in Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 359–368, Seattle, WA, USA, November 2018.

[38]

J. Xu, L. Chen, M. Lv, C. Zhan, S. Chen, and J. Chang, “Highair: a hierarchical graph neural network-based air quality forecasting method,” 2021, https://arxiv.org/abs/2101.04264.

[39]

C. Yu, X. Ma, J. Ren, H. Zhao, and S. Yi, “Spatio-temporal graph transformer networks for pedestrian trajectory prediction,” in Proceedings of the European Conference on Computer Vision, pp. 507–523, Berlin, Germany, May 2020.

[40]

S. P. Arango, F. Heinrich, K. Madhusudhanan, and L. Schmidt-Thieme, “Multimodal meta-learning for time series regression,” in Proceedings of the International Workshop on Advanced Analytics and Learning on Temporal Data, pp. 123–138, Berlin, Germany, August 2021.

[41]

T. Li, X. Su, W. Liu, W. Liang, M.-Y. Hsieh, Z. Chen, X. Liu, and H. Zhang, “Memory-augmented meta-learning on meta-path for fast adaptation cold-start recommendation,” Connection Science, vol. 34, no. 1, pp. 301–318, 2022.

[42]

S. Tariq, J. Loy-Benitez, K. Nam, G. Lee, M. Kim, D. Park, and C. Yoo, “Transfer learning driven sequential forecasting and ventilation control of pm2. 5 associated health risk levels in underground public facilities,” Journal of Hazardous Materials, vol. 406, 2021.

[43]

Z. Wu, C. Ma, X. Shi, L. Wu, D. Zhang, Y. Tang, and M. Stojmenovic, “Brnn-gan: generative adversarial networks with bi-directional recurrent neural networks for multivariate time series imputation,” in Proceedings of the 2021 IEEE 27th International Conference on Parallel and Distributed Systems (ICPADS), pp. 217–224, Beijing, China, December 2021.

[44]

Z. Wu, C. Ma, X. Shi, L. Wu, Y. Dong, and M. Stojmenovic, “Imputing missing indoor air quality data with inverse mapping generative adversarial network,” Building and Environment, vol. 215, 2022.

[45]

S. Thrun and L. Pratt, “Learning to learn: introduction and overview,” Learning to Learn, vol. 78, pp. 120–193, 1998.

[46]

C. Finn, A. Rajeswaran, S. Kakade, and S. Levine, “Online meta-learning,” in Proceedings of the International Conference on Machine Learning, pp. 1920–1930, Baltimore, MA, USA, September 2019.

[47]

D. Li, Y. Yang, Y.-Z. Song, and T. Hospedales, “Learning to generalize: meta-learning for domain generalization,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018.

[48]

J. R. Schwarz and Y. W. Teh, “Meta-learning sparse compression networks,” 2022, https://arxiv.org/abs/2205.08957.

[49]

A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap, “Meta-learning with memory-augmented neural networks,” in Proceedings of the International Conference on Machine Learning, pp. 1842–1850, Baltimore, MA, USA, May 2016.

[50]

H. Kim, A. Mnih, J. Schwarz, M. Garnelo, A. Eslami, D. Rosenbaum, O. Vinyals, and Y. W. Teh, “Attentive neural processes,” 2019, https://arxiv.org/abs/1901.05761.

[51]

C. H. Kao, W.-C. Chiu, and P.-Y. Chen, “Maml is a noisy contrastive learner in classification,” in Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia, August 2021.

[52]

J. Snell, K. Swersky, and R. S. Zemel, “Prototypical networks for few-shot learning,” 2017, https://arxiv.org/abs/1703.05175.

[53]

K. Lee, S. Maji, A. Ravichandran, and S. Soatto, “Meta-learning with differentiable convex optimization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10657–10665, Long Beach, CA, USA, June 2019.

[54]

A. Nichol, J. Achiam, and J. Schulman, “On first-order meta-learning algorithms,” 2018, https://arxiv.org/abs/1803.02999.

[55]

A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, R. Pascanu, S. Osindero, and R. Hadsell, “Meta-learning with latent embedding optimization,” 2018, https://arxiv.org/abs/1807.05960.

[56]

J. Xu, J.-F. Ton, H. Kim, A. Kosiorek, and Y. W. Teh, “Metafun: meta-learning with iterative functional updates,” in Proceedings of the International Conference on Machine Learning, Beijing, China, November 2020.

[57]

T. Deleu, D. Kanaa, L. Feng, G. Kerg, Y. Bengio, G. Lajoie, and P.-L. Bacon, “Continuous-time meta-learning with forward mode differentiation,” 2022, https://arxiv.org/abs/2203.01443.

[58]

H. Edwards and A. Storkey, “Towards a neural statistician,” 2016, https://arxiv.org/abs/1606.02185.

[59]

E. Grant, C. Finn, S. Levine, T. Darrell, and T. Griffiths, “Recasting gradient-based meta-learning as hierarchical bayes,” 2018, https://arxiv.org/abs/1801.08930.

[60]

C. Finn, K. Xu, and S. Levine, “Probabilistic model-agnostic meta-learning,” in Proceedings of the 32nd International Conference On Neural Information Processing Systems, pp. 9537–9548, Red Hook, NY, USA, June 2018.

[61]

S. Ravi and A. Beatson, “Amortized bayesian meta-learning,” in Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia, May 2018.

[62]

J. Yoon, T. Kim, O. Dia, S. Kim, Y. Bengio, and S. Ahn, “Bayesian model-agnostic meta-learning,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 7343–7353, Red Hook, NY, USA, December 2018.

[63]

Z. Pan, W. Zhang, Y. Liang, W. Zhang, Y. Yu, J. Zhang, and Y. Zheng, “Spatio-temporal meta learning for urban traffic prediction,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, 2020.

[64]

M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. Salakhutdinov, and A. Smola, “Deep sets,” 2017, https://arxiv.org/abs/1703.06114.

[65]

X. Zhang, Y. Li, X. Zhou, O. Mangoubi, Z. Zhang, V. Filardi, and J. Luo, “Dac-ml: domain adaptable continuous meta-learning for urban dynamics prediction,” in Proceedings of the 2021 IEEE International Conference on Data Mining (ICDM), pp. 906–915, Shanghai, China, June 2021.

[66]

J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh, “Set transformer: a framework for attention-based permutation-invariant neural networks,” in Proceedings of the International Conference on Machine Learning, pp. 3744–3753, Long Beach, CA, USA, June 2019.

[67]

R. J. Santos, “Equivalence of regularization and truncated iteration for general ill-posed problems,” Linear Algebra and Its Applications, vol. 236, pp. 25–33, 1996.

[68]

N. Gao, H. Ziesche, N. A. Vien, M. Volpp, and G. Neumann, “What matters for meta-learning vision regression tasks?” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14776–14786, Washington, DC, USA, November 2022.

[69]

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, 2017.

Recommendations

Urban Air Pollution Monitoring by Ground-Based Stations and Satellite Data: Multi-season characteristics from Lanzhou City, China
Wind-sensitive Interpolation of Urban Air Pollution Forecasts

People living in urban areas are exposed to outdoor air pollution. Air contamination is linked to numerous premature and pre-native deaths each year. Urban air pollution is estimated to cost approximately 2% of GDP in developed countries and 5% in ...
Air pollution in the area around the mining complex Trepça in Kosovo
EE'10: Proceedings of the 5th IASME/WSEAS international conference on Energy & environment

In this paper are presented the results of air pollution in Mining Complex Trepça in Kosovo. The region is significantly affected by industrial pollution which is affected by the mining sector. Associated to the region there is effluents by toxic/acidic ...

Comments

Information & Contributors

Information

Published In

International Journal of Intelligent Systems Volume 2023, Issue

2023

3141 pages

ISSN:0884-8173

Issue’s Table of Contents

Copyright © 2023 Zhiyuan Wu et al.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Publisher

John Wiley and Sons Ltd.

United Kingdom

Publication History

Published: 01 January 2023

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Sep 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents