Mobility Enhancement & Vision Rehabilitation

This paper proposes a bio-inspired visual motion estimation algorithm based on motion energy, along with its compact very-large-scale integration (VLSI) architecture using low-cost embedded systems. The algorithm mimics motion perception functions of retina, V1, and MT neurons in a primate visual system. It involves operations of ternary edge extraction, spatiotemporal filtering, motion energy extraction, and velocity integration. Moreover, we propose the concept of confidence map to indicate the reliability of estimation results on each probing location. Our algorithm involves only additions and multiplications during runtime, which is suitable for low-cost hardware implementation. The proposed VLSI architecture employs multiple (frame, pixel, and operation) levels of pipeline and massively parallel processing arrays to boost the system performance. The array unit circuits are optimized to minimize hardware resource consumption. We have prototyped the proposed architecture on a low-cost field-programmable gate array platform (Zynq 7020) running at 53-MHz clock frequency. It achieved 30-frame/s real-time performance for velocity estimation on 160 × 120 probing locations. A comprehensive evaluation experiment showed that the estimated velocity by our prototype has relatively small errors (average endpoint error < 0.5 pixel and angular error < 10°) for most motion cases.

A normally sighted person can see a grating of 30 cycles per degree or higher, but spatial frequencies needed for motion perception are much lower than that. It is unknown for natural images with a wide spectrum how all the visible spatial frequencies contribute to motion speed perception. In this work, we studied the effect of spatial frequency content on motion speed estimation for sequences of natural and stochastic pixel images by simulating different visual conditions, including normal vision, low vision (low-pass filtering), and complementary vision (high-pass filtering at the same cutoff frequencies of the corresponding low-vision conditions) conditions. Speed was computed using a biological motion energy-based computational model. In natural sequences, there was no difference in speed estimation error between normal vision and low vision conditions, but it was significantly higher for complementary vision conditions (containing only high-frequency components) at higher speeds. In stochastic sequences that had a flat frequency distribution, the error in normal vision condition was significantly larger compared with low vision conditions at high speeds. On the contrary, such a detrimental effect on speed estimation accuracy was not found for low spatial frequencies. The simulation results were consistent with the motion direction detection task performed by human observers viewing stochastic sequences. Together, these results (i) reiterate the importance of low frequencies in motion perception, and (ii) indicate that high frequencies may be detrimental for speed estimation when low frequency content is weak or not present.

Optomotor response/reflex (OMR) assays are emerging as a powerful and versatile tool for phenotypic study and new drug discovery for eye and brain disorders. Yet efficient OMR assessment for visual performance in mice remains a challenge. Existing OMR testing devices for mice require a lengthy procedure and may be subject to bias due to use of artificial criteria. We developed an optimized staircase protocol that utilizes mouse head pausing behavior as a novel indicator for the absence of OMR, to allow rapid and unambiguous vision assessment. It provided a highly sensitive and reliable method that can be easily implemented into automated or manual OMR systems to allow quick and unbiased assessment for visual acuity and contrast sensitivity in mice. The sensitivity and quantitative capacity of the protocol were validated using wild type mice and an inherited mouse model of retinal degeneration - mice carrying rhodopsin deficiency and exhibiting progressive loss of photoreceptors. Our OMR system with this protocol was capable of detecting progressive visual function decline that was closely correlated with the loss of photoreceptors in rhodopsin deficient mice. It provides significant advances over the existing methods in the currently available OMR devices in terms of sensitivity, accuracy and efficiency.

A growing body of evidence demonstrates that the brain can reorganize dramatically following sensory loss. Although the existence of such neuroplastic crossmodal changes is not in doubt, the functional significance of these changes remains unclear. The dominant belief is that reorganization is compensatory. However, results thus far do not unequivocally indicate that sensory deprivation results in markedly enhanced abilities in other senses. Here, we consider alternative reasons besides sensory compensation that might drive the brain to reorganize after sensory loss. One such possibility is that the cortex reorganizes not to confer functional benefits, but to avoid undesirable physiological consequences of sensory deafferentation. Empirical assessment of the validity of this and other possibilities defines a rich program for future research.

Eye and head movements are used to scan the environment when driving. In particular, when approaching an intersection, large gaze scans to the left and right, comprising head and multiple eye movements, are made. We detail an algorithm called the gaze scan algorithm that automatically quantifies the magnitude, duration, and composition of such large lateral gaze scans. The algorithm works by first detecting lateral saccades, then merging these lateral saccades into gaze scans, with the start and end points of each gaze scan marked in time and eccentricity. We evaluated the algorithm by comparing gaze scans generated by the algorithm to manually marked "consensus ground truth" gaze scans taken from gaze data collected in a high-fidelity driving simulator. We found that the gaze scan algorithm successfully marked 96% of gaze scans and produced magnitudes and durations close to ground truth. Furthermore, the differences between the algorithm and ground truth were similar to the differences found between expert coders. Therefore, the algorithm may be used in lieu of manual marking of gaze data, significantly accelerating the time-consuming marking of gaze movement data in driving simulator studies. The algorithm also complements existing eye tracking and mobility research by quantifying the number, direction, magnitude, and timing of gaze scans and can be used to better understand how individuals scan their environment.

Purpose: One rehabilitation strategy taught to individuals with hemianopic field loss (HFL) is to make a large blind side scan to quickly identify hazards. However, it is not clear what the minimum threshold is for how large the scan should be. Using driving simulation, we evaluated thresholds (criteria) for gaze and head scan magnitudes that best predict detection safety. Methods: Seventeen participants with complete HFL and 15 with normal vision (NV) drove through 4 routes in a virtual city while their eyes and head were tracked. Participants pressed the horn as soon as they detected a motorcycle (10 per drive) that appeared 54 degrees eccentricity on cross-streets and approached toward the driver. Results: Those with HFL detected fewer motorcycles than those with NV and had worse detection on the blind side than the seeing side. On the blind side, both safe detections and early detections (detections before the hazard entered the intersection) could be predicted with both gaze (safe 18.5 degrees and early 33.8 degrees) and head (safe 19.3 degrees and early 27 degrees) scans. However, on the seeing side, only early detections could be classified with gaze (25.3 degrees) and head (9.0 degrees). Conclusions: Both head and gaze scan magnitude were significant predictors of detection on the blind side, but less predictive on the seeing side, which was likely driven by the ability to use peripheral vision. Interestingly, head scans were as predictive as gaze scans. Translational Relevance: The minimum scan magnitude could be a useful criterion for scanning training or for developing assistive technologies to improve scanning.

Natural vision often involves recognizing objects from partial information. Recognition of objects from parts presents a significant challenge for theories of vision because it requires spatial integration and extrapolation from prior knowledge. Here we recorded intracranial field potentials of 113 visually selective electrodes from epilepsy patients in response to whole and partial objects. Responses along the ventral visual stream, particularly the inferior occipital and fusiform gyri, remained selective despite showing only 9%-25% of the object areas. However, these visually selective signals emerged ∼100 ms later for partial versus whole objects. These processing delays were particularly pronounced in higher visual areas within the ventral stream. This latency difference persisted when controlling for changes in contrast, signal amplitude, and the strength of selectivity. These results argue against a purely feedforward explanation of recognition from partial information, and provide spatiotemporal constraints on theories of object recognition that involve recurrent processing.

We introduce a new task for exploring the relationship between action and attention. In this interactive multiple object tracking (iMOT) task, implemented as an iPad app, participants were presented with a display of multiple, visually identical disks which moved independently. The task was to prevent any collisions during a fixed duration. Participants could perturb object trajectories via the touchscreen. In Experiment 1, we used a staircase procedure to measure the ability to control moving objects. Object speed was set to 1°/s. On average participants could control 8.4 items without collision. Individual control strategies were quite variable, but did not predict overall performance. In Experiment 2, we compared iMOT with standard MOT performance using identical displays. Object speed was set to 2°/s. Participants could reliably control more objects (M = 6.6) than they could track (M = 4.0), but performance in the two tasks was positively correlated. In Experiment 3, we used a dual-task design. Compared to single-task baseline, iMOT performance decreased and MOT performance increased when the two tasks had to be completed together. Overall, these findings suggest: 1) There is a clear limit to the number of items that can be simultaneously controlled, for a given speed and display density; 2) participants can control more items than they can track; 3) task-relevant action appears not to disrupt MOT performance in the current experimental context.

PURPOSE: Examine the effect of force modulation via angular translation of a static magnetic field for customizable treatment of severe blepharoptosis. METHODS: Prototype adjustable-force magnetic levator prostheses (aMLP) consisted of a spectacle-mounted magnet in rotatable housing and small eyelid-attached magnets embedded in a biocompatible polymer. Interpalpebral fissure (IPF) of 17 participants with severe blepharoptosis was continuously measured for one minute at five spectacle magnet angles, with order randomized and participant and data analyst masked. The hypothesis that angular position affected opening IPF (o-IPF), minimum blink IPF (m-IPF), and comfort ratings (1-10) was tested. RESULTS: The aMLP improved o-IPF from 4.5 mm without the device to 6.2 mm on the lowest force setting (P < 0.001) and 7.1 mm on the highest setting (P < 0.001) and allowed for complete volitional blink regardless of setting (average m-IPF 0.4 mm and no change with aMLP; P = 0.76). Spontaneous blink without the device (2.0 mm) was affected on the highest force setting (m-IPF 3.9 mm; P < 0.001) but only marginally so on the lowest setting (3.0 mm; P = 0.06). Comfort (7.6/10) did not vary with the angle (P > 0.36). Profile analysis found substantial individual responses to angle (P < 0.001), confirming the value of customization. CONCLUSIONS: Angular translation provided adjustable force, which had a statistically and clinically meaningful impact on eye opening and the completeness of the spontaneous blink. This quantitative evidence supports continued use of the angular translation mechanism for force adjustment in the customizable magnetic correction of severe blepharoptosis. TRANSLATIONAL RELEVANCE: Evidence for the benefit of customizable magnetic force via angular translation in a larger sample of participants than reported previously.

IMPORTANCE: There are limited reports on the relationship between mucin 5AC (MUC5AC) concentrations in tears, working hours, and the frequency of ocular symptoms in visual display terminal (VDT) users. This investigation evaluated these relationships among patients with dry eye disease (DED) and individuals serving as controls. OBJECTIVE: To determine the relationship between MUC5AC concentration in the tears of VDT users based on the diagnosis of DED and frequency of ocular symptoms. DESIGN, SETTING, AND PARTICIPANTS: An institutional, cross-sectional study was conducted. Participants included 96 young and middle-aged Japanese office workers. Both eyes of 96 volunteers (60 men and 36 women) were studied. Participants working in a company that used VDTs completed questionnaires about their working hours and the frequency of ocular symptoms. Dry eye disease was diagnosed as definite or probable, or it was not present. Tear fluid was collected from the inferior fornix after instillation of 50 μL of sterilized saline. The MUC5AC concentration was normalized to tear protein content and expressed as MUC5AC (nanograms) per tear protein (milligrams). The differences in MUC5AC concentration between DED groups, between VDT working hours (short, intermediate, and long), and between symptomatic and asymptomatic groups were evaluated with 95% CIs based on nonparametric Hodges-Lehmann determination. MAIN OUTCOMES AND MEASURES: Ocular surface evaluation, prevalence of DED, and MUC5AC concentration. RESULTS: The prevalence of definite and probable DED was 9% (n = 9) and 57% (n = 55), respectively. The mean MUC5AC concentration was lower in the tears of VDT users with definite DED than in those with no DED (P = .02; Hodges-Lehmann estimator, -2.17; 95% CI, -4.67 to -0.30). The mean MUC5AC concentration in tears was lower in the group that worked longer hours than in the group that worked shorter hours (P = .049; estimated difference, -1.65; 95% CI, -3.12 to 0.00). Furthermore, MUC5AC concentration was lower in participants with symptomatic eye strain than in asymptomatic individuals (P = .001; estimated difference, -1.71; 95% CI, -2.86 to -0.63). CONCLUSIONS AND RELEVANCE: The data obtained in the present study suggest that office workers with prolonged VDT use, as well as those with an increased frequency of eye strain, have a low MUC5AC concentration in their tears. Furthermore, MUC5AC concentration in the tears of patients with DED may be lower than that in individuals without DED.

We investigated the relative influence of image salience and image semantics during the visual search of naturalistic scenes, comparing performance in individuals with cerebral visual impairment (CVI) and controls with neurotypical development. Participants searched for a prompted target presented as either an image or text cue. Success rate and reaction time were collected, and gaze behavior was recorded with an eye tracker. A receiver operating characteristic (ROC) analysis compared the distribution of individual gaze landings based on predictions of image salience (using Graph-Based Visual Saliency) and image semantics (using Global Vectors for Word Representations combined with Linguistic Analysis of Semantic Salience) models. CVI participants were less likely and were slower in finding the target. Their visual search behavior was also associated with a larger visual search area and greater number of fixations. ROC scores were also lower in CVI compared to controls for both model predictions. Furthermore, search strategies in the CVI group were not affected by cue type, although search times and accuracy showed a significant correlation with verbal IQ scores for text-cued searches. These results suggest that visual search patterns in CVI are driven mainly by image salience and provide further characterization of higher-order processing deficits observed in this population.

Gaze-contingent displays have been widely used in vision research and virtual reality applications. Due to data transmission, image processing, and display preparation, the time delay between the eye tracker and the monitor update may lead to a misalignment between the eye position and the image manipulation during eye movements. We propose a method to reduce the misalignment using a Taylor series to predict the saccadic eye movement. The proposed method was evaluated using two large datasets including 219,335 human saccades (collected with an EyeLink 1000 system, 95% range from 1° to 32°) and 21,844 monkey saccades (collected with a scleral search coil, 95% range from 1° to 9°). When assuming a 10-ms time delay, the prediction of saccade movements using the proposed method could reduce the misalignment greater than the state-of-the-art methods. The average error was about 0.93° for human saccades and 0.26° for monkey saccades. Our results suggest that this proposed saccade prediction method will create more accurate gaze-contingent displays.

We tested younger and older observers' attention and long-term memory functions in a "hybrid search" task, in which observers look through visual displays for instances of any of several types of targets held in memory. Apart from a general slowing, search efficiency did not change with age. In both age groups, reaction times increased linearly with the visual set size and logarithmically with the memory set size, with similar relative costs of increasing load (Experiment 1). We replicated the finding and further showed that performance remained comparable between age groups when familiarity cues were made irrelevant (Experiment 2) and target-context associations were to be retrieved (Experiment 3). Our findings are at variance with theories of cognitive aging that propose age-specific deficits in attention and memory. As hybrid search resembles many real-world searches, our results might be relevant to improve the ecological validity of assessing age-related cognitive decline.

The prevalence and reward-value of targets have an influence on visual search. The strength of the effect of an item's reward-value on attentional selection varies substantially between individuals and is potentially sensitive to aging. We investigated individual and age differences in a hybrid foraging task, in which the prevalence and value of multiple target types was varied. Using optimal foraging theory measures, foraging was more efficient overall in younger than older observers. However, the influence of prevalence and value on target selections was similar across age groups, suggesting that the underlying cognitive mechanisms are preserved in older age. When prevalence was varied but target value was balanced, younger and older observers preferably selected the most frequent target type and were biased to select another instance of the previously selected target type. When value was varied, younger and older observers showed a tendency to select high-value targets, but preferences were more diverse between individuals. When value and prevalence were inversely related, some observers showed particularly strong preferences for high-valued target types, while others showed a preference for high-prevalent, albeit low-value, target types. In younger adults, individual differences in the selection choices correlated with a personality index, suggesting that avoiding selections of low-value targets may be related to reward-seeking behaviour.

Sequence learning effects in simple perceptual and motor tasks are largely unaffected by normal aging. However, less is known about sequence learning in more complex cognitive tasks that involve attention and memory processes and how this changes with age. In this study, we examined whether incidental and intentional sequence learning would facilitate hybrid visual and memory search in younger and older adults. Observers performed a hybrid search task, in which they memorized four or 16 target objects and searched for any of those target objects in displays with four or 16 objects. The memorized targets appeared either in a repeating sequential order or in random order. In the first experiment, observers were not told about the sequence before the experiment. Only a subset of younger adults and none of the older adults incidentally learned the sequence. The "learners" acquired explicit knowledge about the sequence and searched faster in the sequence compared to random condition. In the second experiment, observers were told about the sequence before the search task. Both younger and older adults searched faster in sequence blocks than random blocks. Older adults, however, showed this sequence-learning effect only in blocks with smaller target sets. Our findings indicate that explicit sequence knowledge can facilitate hybrid search, as it allows observers to predict the next target and restrict their visual and memory search. In older age, the sequence-learning effect is constrained by load, presumably due to age-related decline in executive functions.

It is well known that humans have a massive memory for pictures and scenes.1,2,3,4 They show an ability to encode thousands of images with only a few seconds of exposure to each. In addition to this massive memory for "what" observers have seen, three experiments reported here show that observers have a "spatial massive memory" (SMM) for "where" stimuli have been seen and a "temporal massive memory" (TMM) for "when" stimuli have been seen. The positions in time and space for at least dozens of items can be reported with good, if not perfect accuracy. Previous work has suggested that there might be good memory for stimulus location,5,6 but there do not seem to have been concerted efforts to measure the extent of this memory. Moreover, in our method, observers are recalling where items were located and not merely recognizing the correct location. This is interesting because massive memory is sometimes thought to be limited to recognition tasks based on sense of familiarity.