Alinaghi, Negar; Giannopoulos, Ioannis Consider the Head Movements! Saccade Computation in Mobile Eye-Tracking (Inproceedings) In: 2022 Symposium on Eye Tracking Research and Applications, Association for Computing Machinery, Seattle, WA, USA, 2022, ISBN: 9781450392525. @inproceedings{10.1145/3517031.3529624,
title = {Consider the Head Movements! Saccade Computation in Mobile Eye-Tracking},
author = {Negar Alinaghi and Ioannis Giannopoulos},
url = {https://doi.org/10.1145/3517031.3529624},
doi = {10.1145/3517031.3529624},
isbn = {9781450392525},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 Symposium on Eye Tracking Research and Applications},
publisher = {Association for Computing Machinery},
address = {Seattle, WA, USA},
series = {ETRA '22},
abstract = {Saccadic eye movements are known to serve as a suitable proxy for tasks prediction. In mobile eye-tracking, saccadic events are strongly influenced by head movements. Common attempts to compensate for head-movement effects either neglect saccadic events altogether or fuse gaze and head-movement signals measured by IMUs in order to simulate the gaze signal at head-level. Using image processing techniques, we propose a solution for computing saccades based on frames of the scene-camera video. In this method, fixations are first detected based on gaze positions specified in the coordinate system of each frame, and then respective frames are merged. Lastly, pairs of consecutive fixations –forming a saccade- are projected into the coordinate system of the stitched image using the homography matrices computed by the stitching algorithm. The results show a significant difference in length between projected and original saccades, and approximately 37% of error introduced by employing saccades without head-movement consideration.},
keywords = {AI, eye tracking, Saccades},
pubstate = {published},
tppubtype = {inproceedings}
}
Saccadic eye movements are known to serve as a suitable proxy for tasks prediction. In mobile eye-tracking, saccadic events are strongly influenced by head movements. Common attempts to compensate for head-movement effects either neglect saccadic events altogether or fuse gaze and head-movement signals measured by IMUs in order to simulate the gaze signal at head-level. Using image processing techniques, we propose a solution for computing saccades based on frames of the scene-camera video. In this method, fixations are first detected based on gaze positions specified in the coordinate system of each frame, and then respective frames are merged. Lastly, pairs of consecutive fixations –forming a saccade- are projected into the coordinate system of the stitched image using the homography matrices computed by the stitching algorithm. The results show a significant difference in length between projected and original saccades, and approximately 37% of error introduced by employing saccades without head-movement consideration. |
Cutchan, Marvin Mc; Giannopoulos, Ioannis Encoding Geospatial Vector Data for Deep Learning: LULC as a Use Case (Journal Article) In: Remote Sensing, vol. 14, no. 12, 2022, ISSN: 2072-4292. @article{rs14122812,
title = {Encoding Geospatial Vector Data for Deep Learning: LULC as a Use Case},
author = {Marvin Mc Cutchan and Ioannis Giannopoulos},
url = {https://www.mdpi.com/2072-4292/14/12/2812},
doi = {10.3390/rs14122812},
issn = {2072-4292},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Remote Sensing},
volume = {14},
number = {12},
abstract = {Geospatial vector data with semantic annotations are a promising but complex data source for spatial prediction tasks such as land use and land cover (LULC) classification. These data describe the geometries and the types (i.e., semantics) of geo-objects, such as a Shop or an Amenity. Unlike raster data, which are commonly used for such prediction tasks, geospatial vector data are irregular and heterogenous, making it challenging for deep neural networks to learn based on them. This work tackles this problem by introducing novel encodings which quantify the geospatial vector data allowing deep neural networks to learn based on them, and to spatially predict. These encodings were evaluated in this work based on a specific use case, namely LULC classification. We therefore classified LULC based on the different encodings as input and an attention-based deep neural network (called Perceiver). Based on the accuracy assessments, the potential of these encodings is compared. Furthermore, the influence of the object semantics on the classification performance is analyzed. This is performed by pruning the ontology, describing the semantics and repeating the LULC classification. The results of this work suggest that the encoding of the geography and the semantic granularity of geospatial vector data influences the classification performance overall and on a LULC class level. Nevertheless, the proposed encodings are not restricted to LULC classification but can be applied to other spatial prediction tasks too. In general, this work highlights that geospatial vector data with semantic annotations is a rich data source unlocking new potential for spatial predictions. However, we also show that this potential depends on how much is known about the semantics, and how the geography is presented to the deep neural network.},
keywords = {AI, Deep learning, geoinformation, geosemantics, LULC, Volunteered Geographic Information},
pubstate = {published},
tppubtype = {article}
}
Geospatial vector data with semantic annotations are a promising but complex data source for spatial prediction tasks such as land use and land cover (LULC) classification. These data describe the geometries and the types (i.e., semantics) of geo-objects, such as a Shop or an Amenity. Unlike raster data, which are commonly used for such prediction tasks, geospatial vector data are irregular and heterogenous, making it challenging for deep neural networks to learn based on them. This work tackles this problem by introducing novel encodings which quantify the geospatial vector data allowing deep neural networks to learn based on them, and to spatially predict. These encodings were evaluated in this work based on a specific use case, namely LULC classification. We therefore classified LULC based on the different encodings as input and an attention-based deep neural network (called Perceiver). Based on the accuracy assessments, the potential of these encodings is compared. Furthermore, the influence of the object semantics on the classification performance is analyzed. This is performed by pruning the ontology, describing the semantics and repeating the LULC classification. The results of this work suggest that the encoding of the geography and the semantic granularity of geospatial vector data influences the classification performance overall and on a LULC class level. Nevertheless, the proposed encodings are not restricted to LULC classification but can be applied to other spatial prediction tasks too. In general, this work highlights that geospatial vector data with semantic annotations is a rich data source unlocking new potential for spatial predictions. However, we also show that this potential depends on how much is known about the semantics, and how the geography is presented to the deep neural network. |