[AHW37-P02] Development of climate change-related data accumulation system in Thailand
Keywords:climate change, open access, data-sharing
Although certain data may be available to researchers, accessing that data in a usable form is not always straightforward, especially for researchers without a programming background. Some data, such as weather observations, are accessible only for a short time. Historical datasets are usually not easy to download due to the large file sizes, which tend to consume an organization’s network bandwidth. Although some organizations provide historical data, these are often in a form that is difficult to extract. For example, the Electricity Generating Authority of Thailand (EGAT) allows access to dam-operation data spanning many decades using an interactive web application. Researchers can view the data for a given date, and individual data points can then be copied into a spreadsheet. This procedure can be repeated as many times as needed; however, it is error-prone and time-consuming, especially when a large dataset is required. The Thai Meteorological Department (TMD) and Royal Irrigation Department (RID) use a different approach, by providing Application Programming Interfaces (APIs) that can specify data types and time periods of interest when downloading a dataset. This method saves considerable time and network bandwidth, but still requires a degree of programming knowledge.
To address these shortcomings and facilitate data access for researchers, our project (ADAP-T; Advancing Co-Design of Integrated Strategies with Adaptation to Climate Change in Thailand) has designed a data-sharing architecture. The main servers are responsible for downloading and archiving data in various forms from related organizations. At the time of writing, the ADAP-T project has been archiving weather observations and runoff data from the TMD and RID via their respective APIs. The TMD also publishes hourly QPE (Quantitative precipitation estimation) data, which can also be downloaded and archived. The EGAT, whose dam operation data are required by several groups, does not allow access to raw data, so a web scraping technique must be used. The archived data are then provided to ADAP-T researchers via a unified API, as well as simple comma-separated values (CSV) files. The results produced by different groups are also collated and can be accessed by researchers outside the ADAP-T project, if permitted. Unique results not found on other websites can be visualized in an interactive fashion and viewed by adaptation policymakers and the general public.
To address these shortcomings and facilitate data access for researchers, our project (ADAP-T; Advancing Co-Design of Integrated Strategies with Adaptation to Climate Change in Thailand) has designed a data-sharing architecture. The main servers are responsible for downloading and archiving data in various forms from related organizations. At the time of writing, the ADAP-T project has been archiving weather observations and runoff data from the TMD and RID via their respective APIs. The TMD also publishes hourly QPE (Quantitative precipitation estimation) data, which can also be downloaded and archived. The EGAT, whose dam operation data are required by several groups, does not allow access to raw data, so a web scraping technique must be used. The archived data are then provided to ADAP-T researchers via a unified API, as well as simple comma-separated values (CSV) files. The results produced by different groups are also collated and can be accessed by researchers outside the ADAP-T project, if permitted. Unique results not found on other websites can be visualized in an interactive fashion and viewed by adaptation policymakers and the general public.