Examining The Effect of Conflicts in a Semantic EO DC

Armed conflicts mean devastating loss of civilian life, massive displacement and violations of human rights and international humanitarian law. In addition to these, armed conflicts give rise to economic downturn, loss of livelihoods and large-scale food insecurity.

Quoting Gates et al 2012,

The consequences of war extend far beyond direct deaths. In addition to battlefield casualties, armed conflict often leads to forced migration, refugee flows, capital flight, and the destruction of societies’ infrastructure. It also creates a development gap between those countries that have experienced armed conflict and those that have not.

Agriculture is an important aspect of Aghan economy that has been devastated by Conflict. Apart from foreign aid, Agriculture is the only other source of GDP for the country. Agriculture employs more than 40% of the total population, and upto 60% of the rural workforce. It also contributes about 25% of GDP, and is the most productive sector of the economy. But following its 4 decades of conflict, the agricultural sector has stagnated, as its share of GDP fell from 71% in 1994 to 24% in 2013. Today, farmers are dependent upon imports of both wheat and wheat seed. (FAO)

Specific aspects of explosive violence that directly impact Agriculture negatively;

• Landmines from the 1970 and 80s leaves arable farmland unused and polluted
• Destruction of agricultural structures
• Farmers fear for their lives from air raids and drone strikes, which have in the past targeted farmers unknowingly.
• Many farmers are displaced internally or become refugees in other countries. Therefore their farms lie fallow.

Apart from explosive violence, the effects of natural phenomenon such as prolonged drought on agriculture cannot be underestimated.

It started as a part of the Arab spring. Still ongoing 11 years later. It is particularly notorious for it’s the high death toll of civilians, and the fact that the actions on both sides have been devastating to civilians particularly. This is evident in the fact that Syria’s population of 21 million in 2010 has shrunk to about 17 million in 2020 due to the ongoing conflict (The World Bank 2022).

Specific aspects of explosive violence that directly impact Agriculture negatively;

• Opposing forces (mostly government) directly targeting farmland, causing crop fires, what activists view as the government using hunger as a weapon of war. (this has been widely reported in Syria as well as Iraq).

Majority, if not all EO analysis and visualization tools out there can be applicable in conflict monitoring, however as per the instructions for this course, I will be focusing on using a Semantic EO Data Cube, specifically  Sen2Cube.at .

According to the team behind Sen2Cube, a semantic EO data cube or a semantics-enabled EO data cube is a data cube, where for each observation at least one nominal (i.e. categorical) interpretation is available and can be queried in the same instance. (Augustine et al 2019).

The main purpose behind this, and what sets it apart is the concept of semantic enrichment.

Semantic enrichment is done through Semantic Content Based Image Retrieval (SCBIR) It is achieved through a pre-classification with the Satellite Image Automatic Mapper ™ software (SIAM ™), developed by Baraldi (2019). This is fully automated, done without any user-defined parameterization or training data. This results in categorization which are assigned color names known as semiconcepts.

These semi-concepts are then stored with each image in the data cube thereby resulting in its semantic enrichment. The user can further combine these semantic categorizations with other capabilities (e.g. creating indices or analysis through time and space), using the model builder (knowledge base) to create their own inferences and then arrive at their desired results.

Semantic queries in Sen2Cube.at are composed as follows:

1. First, a semantic model is required which can be created from scratch.
2. Then an AOI is set (subset of the chosen Factbase) followed by a time interval.
3. After processing, one or many results are obtained.
4. Metadata such as the duration of the execution, creator etc. is also provided using the Python library xarray, the Sen2Cube.at team developed the inference engine internally.
5. The inference engine assesses and processes the semantic model that was used for the query. The gathered information is sent back in interoperable data formats for visualization and additional processing.
6. Maps are downloadable as GeoTIFF (or WMS) while Charts can be downloaded as CSV. Also the possibility of exporting as a QGIS Project.

The Sen2Cube only comprehensively covers Austria, and is quite limited in other parts of the world. It currently contains 3 fact-bases other than Austria.

Syria (July 2015 till Jan 2020): Some similar analyses have been done using the Syrian DC. I will discuss one of them later.

Afghanistan Data cube (Jan 2020 to Jan 2022): is limited both spatially (does not cover western Afghanistan, where much of the drought impacts is felt), and temporally - just 2 years.

SemantiX: Advanced Very High Resolution Radiometer Imagery (AVHRR) imagery for Austria and parts of Europe.

Following a similar workflow to the one highlighted above, I tried to analyse possible vegetation change in 3 different AOIs; Bagram (beside the US Airforce base), Nirjab and Charikar.

• Charikar: vegetation gets most of its water form snowmelt and sporadic precipitation. Effects of drought may be visible here, although not as much as in the West.
• Nirjab: same but it also has natural river channels good for irrigation. Vegetation here tends to be lush year round, and is usually not affected by the drought.
• Bagram: for the US Air Force Base that is situated there. It would be good to see how vegetation has changed prior to, but mostly after its use. (The temporal limitation prevented me from performing analysis over a full year since they left.)

First I created a model to estimate the areas where vegetation was observed in the AOI over a month. I applied this to Bagram, Charikar and Nirjab.

Model 1: Observed Vegetation

Semantic Concepts (Entities)

• Vegetation
• Cloud and Snow
• Time line focused on Spring peak planting season (May). So a timeline of April 30th to June 1st is used for each year.

Result

Vegetation entity is masked by cloud and snow and reduced over time in percentage. After getting the results in GeoTIFF form from sen2cube.at, I did further preprocessing and visualization in ArcGIS Pro.

Explanation

The resulting map shows the percentage vegetation, per pixel, recorded over the month. I did this for 2020 and 2021 for all AOIs, presented in comparisons below. However, to show where vegetation was observed generally during the entire month, I styled the resulting map without showing the variation.

In Bagram: The comparison shows increased vegetation overall in 2021, but more in the north western part of the map.

The map below shows for Charikar, where vegetation was observed more in the month in percentage. The light green areas means that through the month analysed, vegetation was present in those areas less than 50% of the time.

When estimating the effects of droughts, these variations are important to show which months and areas where more affected.

Model 2: Vegetation Change Mapping

To better understand the pattern of the change observed in Bagram, I created a second model, the Vegetation Change Model.

Semantic Concepts (Entities)

• Weak Vegetation
• Strong Vegetation
• Cloud and Snow

Results

• Cloud and Snow are reduced over time. I used this to confirm the presence or otherwise of clouds and snow over the image for that period (In sentinel 2 imagery on EO browser, I had observed artefacts that seemed like snow cover in some imagery in May).
• Earlier and Later vegetation (B and A): I combined weak and strong vegetation entities, then masked by cloud and snow entities. I then selected a preferred timeline to be considered (May 2020 and 2021).
• Later minus Earlier: This function subtracts the later from the later vegetation from the earlier.

This model subtracts the 2020 vegetation result from the 2021, to show where vegetation increased or decreased. The map below is for Bagram, where more vegetation change was observed. It confirms what was observed earlier, but also gives hints into where vegetation had reduced, more in the north-east.

I attempted a Smoke plume/Burned area analysis of parts of Syria where explosive violence on agriculture was reported, but it was not successful. I will however discuss my process and possible reasons for this.

Smoke and Burned Area detection Model

The results showed no cloud or burned area detected.

Revised Model in Lower Austria Forest Fire (2021)

Applying the same model as above, but with a new cloud entity and result added (below), to the October 2021 forest fires in Lower Austria, which covers an even larger area (over 100 hectares), I found that although nothing was detected as smoke, or burned area. the smoke plume was actually detected as clouds. So also are highly reflective building roofs and surfaces.

This reveals that the small-scale smoke plumes from active fire, are detected as clouds. This makes sense as conventional active fire and burned area detectors (MODIS, VIIRS, Sentinel 3's SLSTR) employ the use of thermal bands (4nm, 10-12nm) in their sensors to detect hot targets. This is not the case with sen2cube's semantic enrichment of Sentinel 2 imagery, based on semi-concepts that depend only on categorized color names.

Also, with conventional active fire and burned area detectors, the hotter and larger the extent of the fire, the better the detection. This means that cropland fires on this small scale would not be detected. Same goes for burned area mapping, even NASA FIRMS burned area product is also based on MODIS data and thus has the same resolution limitations. A more  recent study  by a NASA Harvest Partner in detecting agricultural burned area in Ukraine used VIRRS 375m active fire product, and found up to 63% underestimation of cropland fires for the MODIS-based product.

When the same updated model is applied to Syria, nothing but buildings are detected as clouds. This reveals that the semantic concepts clouds are influenced by highly reflective surfaces. This is attributable to the software's dependence on only color-based pre-classification for its semantic enrichment.

Although not much research exists testing sen2cube's application for flame or active fire detection, SIAM classifier has proven useful for detecting active fires similar to convention, when applied to MODIS data (as in the image below).

Recommendations

For agricultural fire and burned area estimation from conflict situations or otherwise, in situations where conventional active fire/burned area products may be insufficient due to spatio-temporal limitations, having an alternative with higher spatial resolution such as Sentinel 2 imagery, may prove more useful. Currently, research is being conducted into using Sentinel 2 imagery data for such purposes, e.g. using  SWIR bands  to detect active fires and recently burned areas. If these can in future be considered in sen2cube, then it will prove more applicable in this type of conflict scenario than it is currently.

Semantic EO Data Cubes such as Sen2Cube, although still in its early stages of development, a powerful tool for EO data analytics, as using it has the potential to make EO analysis fun and interactive (model building process) especially for non-programmers, while getting much the same benefits as other major tools out there. One particularly important benefit, I reiterate, is the ability to query by AOI alone. Due to SCBIR, valuable information is not lost due to cloud cover, even for passive sensors like Sentinel 2.

However, there are limitations. Some "current" limitations of the Sen2Cube are:

• Spatial Extent: Currently, the Factbase is limited to 3, with Austria being the only country whose entire extent is contained within it.
• The computational requirement of setting on up is also high, however, there are plans by the team to automate this process (as soon as first quarter 2023). Therefore it will be easier to instantiate data cubes in other regions.
• This issue also cause a limitation for my analysis of the drought affected areas of the western provinces of Afghanistan, which where most affected by the drought.
• The platform is state of the art and relatively small scale, at the moment and therefore may present hitches while processing large extents. It is advised to use small AOIs for now.

Major Updates: to improve functionality and possibly temporal limit (2022 imagery will be back). There is also talks to automate the process of establishing new semantic data cubes, to make it less time consuming. It will soon be easier to set up semantic data cubes for anywhere in the world automatically, which would greatly improve its applicability to current conflict scenarios such as Ukraine.

Gates, S., Hegre H., Nygard H. M. & Strand H. (2012). Development Consequences of Armed Conflict. World Development, 40(9), 1713-1722.

Tiede, Dirk & Sudmanns, Martin & Augustin, Hannah & Lang, Stefan & Baraldi, A.. (2019). Sentinel-2 Semantic Data & Information Cube Austrai. (Link)

Tiede, D.; Lüthje, F.; Baraldi, A. (Eds.) (2014): Automatic post-classification land cover change detection in Landsat images: Analysis of changes in agricultural areas during the Syrian crisis. Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation. Potsdam. DGPF (23).  (Link )

Xikun Hu, Yifang Ban, Andrea Nascetti, Sentinel-2 MSI data for active fire detection in major fire-prone biomes: A multi-criteria approach, International Journal of Applied Earth Observation and Geoinformation, Volume 101, 2021, 102347, ISSN 1569-8432, https://doi.org/10.1016/j.jag.2021.102347. ( Link )

Jan Hofinger (2022) Crisis-Related Agricultural Changes in Northwest Syria: Big EO Data Analysis Using a Sentinel-2 Semantic Data Cube.

Sen2Cube ( Website ,  App ,  Manual )

 FAO, Food Insecurity in Afghanistan