Generic Quality Control Tool for Environmental Marine Datasets

References

[1] Aggarwal, C.C., 2013. Outlier Analysis. Springer.

[2] Bailey, R., Grout, J., Antia, A.N. and Westwood, K., 1994. Quality control procedures for Marinedata. CSIRO Marine Research.

[3] Bettencourt, L.M.A., Hagberg, A. and Larkey, L., 2007. Separating the wheat from the chaff: Practical anomaly detection in environmental data.

[4] Bishop, C.M., 2006. Pattern Recognition and Machine Learning. Springer.

[5] Box, G.E.P., Jenkins, G.M. and Reinsel, G.C., 2008. Time Series Analysis: Forecasting and Control. Wiley.

[6] Castelao, G.P., 2020. A framework to quality control Marine data. Journal of Open Source Software, 5(48), p.2063.

[7] Castro, S.L. and Wick, G.A., 2013. Evaluation of in situ sea surface temperature data quality. Journal of Atmospheric and Oceanic Technology, 30(10), pp.2344–2356.

[8] Chandola, V., Banerjee, A. and Kumar, V., 2009. Anomaly detection: A survey. ACM Computing Surveys, 41(3).

[9] Chatfield, C., 2004. The Analysis of Time Series. CRC Press.

[10] Doong, D.J., Chen, S.H. and Chen, Y.C., 2007. Data quality check procedures of an operational coastal ocean monitoring network. Ocean Engineering, 34(17–18), pp.2349–2360.

[11] Domingues, C.M. and Palmer, M.D., 2015. Ocean data quality and its impact on climate studies.

[12] Durbin, J. and Koopman, S.J., 2012. Time Series Analysis by State Space Methods. Oxford University Press.

[13] Durre, I., Menne, M.J. and Vose, R.S., 2010. Strategies for quality assurance of climate data. Journal of Applied Meteorology and Climatology, 49(5), pp.1004–1016.

[14] Dunn, R.J.H., Willett, K.M., Thorne, P.W. and Woolley, E.V., 2012. HadISD: A quality-controlled global synoptic report dataset.

[15] Emery, W.J. and Thomson, R.E., 2001. Data Analysis Methods in Physical Oceanography. Elsevier.

[16] Fernandes, F., 2021. A machine learning approach to quality control Marine data. Computers & Geosciences, 155, p.104803.

[17] Garcia, H.E., et al., 2018. World Ocean Database 2018: Quality control procedures. NOAA Atlas NESDIS.

[18] Good, S.A., Mills, B., Boyer, T. and Coward, A.C., 2023. Benchmarking of automatic quality control checks for ocean temperature profiles. Frontiers in Marine Science, 9, p.1075510.

[19] Gouretski, V., 2018. Improved methods for oceanographic data quality control.

[20] Hastie, T., Tibshirani, R. and Friedman, J., 2009. The Elements of Statistical Learning. Springer.

[21] Hill, D.J. and Minsker, B., 2010. Anomaly detection in streaming environmental sensor data. Environmental Modelling & Software, 25(9), pp.1014–1026.

[22] Hyndman, R.J. and Athanasopoulos, G., 2018. Forecasting: Principles and Practice.

[23] Ingleby, B. and Huddleston, M., 2007. Quality control of ocean temperature and salinity profiles. Journal of Marine Systems, 65(1–4), pp.158–175.

[24] IOC/IODE, 2010. Manuals and guides for oceanographic data quality control. UNESCO.

[25] Iwaniak, M., et al., 2025. Quality control of time-series seawater temperature and wave data. Oceanological and Hydrobiological Studies.

[26] Keogh, E., Lonardi, S. and Ratanamahatana, C.A., 2004. Towards parameter-free data mining.

[27] Killick, R., Fearnhead, P. and Eckley, I.A., 2012. Optimal detection of changepoints. Journal of the American Statistical Association.

[28] Li, X., et al., 2025. CODC-S: A global ocean salinity dataset with quality control. Scientific Data.

[29] Montgomery, D.C., 2009. Introduction to Statistical Quality Control. Wiley.

[30] Morello, E.B., et al., 2014. Quality control procedures for Australia’s Integrated Marine Observing System.

[31] Palmer, M.D., et al., 2018. The International Quality-Controlled Ocean Database (IQuOD).

[32] Rayner, N.A., et al., 2003. Global analyses of sea surface temperature. Journal of Geophysical Research.

[33] Reynolds, R.W., et al., 2002. Improved global sea surface temperature analyses. Journal of Climate.

[34] Shumway, R.H. and Stoffer, D.S., 2017. Time Series Analysis and Its Applications. Springer.

[35] Smith, T.M., Reynolds, R.W., Peterson, T.C. and Lawrimore, J., 2008. Improvements to NOAA SST datasets.

[36] Sugiura, N. and Hosoda, S., 2020. Statistical approaches for ocean data quality control.

[37] Truong, C., Oudre, L. and Vayatis, N., 2020. Selective review of offline change point detection methods. Signal Processing.

[38] Wilks, D.S., 2011. Statistical Methods in the Atmospheric Sciences. Academic Press.

[39] Wong, A., Keeley, R., Carval, T. and Argo Data Management Team, 2015. Argo quality control manual.

[40] Xu, Y., et al., 2022. A new automatic quality control system for ocean profile observations. Ocean Modelling.

[41] Yuan, L., 2023. Development of a new ocean data quality control system.

[42] Zhang, Y., et al., 2016. Anomaly detection in environmental time series data.

@article{1719223,
      author = {Dr. A. Yaswanth Kumar, D. Raj Kumar},
      title = {Generic Quality Control Tool for Environmental Marine Datasets},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {9},
      number = {12},
      pages = {2737-2747},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1719223.pdf},
      abstract = {In order to ensure the reliability and accuracy of Marine observation data, there is a need to make sure that the data are reliable and of good quality. This is important because in situ datasets obtained via observation platforms are usually subject to measurement errors, faulty instruments, failed communications, and environmental disturbances, which leads to data corruption and misinterpretation. In this paper, a Generic Quality Control Tool for Marine Observation Datasets that automatically determines anomalies in time series is proposed. The framework incorporates several quality control methods, including range testing, spike detection, and stuck value determination. It is implemented in Python language, with the use of Pandas library. The system assigns each record a quality flag depending on the set validation criteria and helps in distinguishing invalid observations from valid ones. The system can be easily customized for different types of environmental variables, including sea surface temperature, salinity, and wave height. The experimental results show the effectiveness of the proposed tool. The proposed system provides a scalable and efficient tool for quality assessment on an automated basis, which will help to increase the reliability of data and environmental monitoring systems.},
      month = {June},
      doi = {https://doi.org/10.64388/IREV9I12-1719223}
  }