Researchers from Shanghai Jiao Tong University, Hefei University of Technology, Korea Advanced Institute of Science and Technology and Tianjin Jinhang Computing Technology Research Institute have developed a retinomorphic hardware prototype that uses a 4096-pixel perovskite image sensor array as core module to endow embodied intelligent vision functionalities.
(A) Adaptive sensing-perception by the biological visual system through the hierarchy of photoreceptors, horizontal cells, bipolar cells, amacrine cells, and ganglion cells. (B) Schematic illustration and working principle of the retinomorphic computing system based on a 4096-pixel 1T-1PD perovskite RSA, microcontroller unit (MCU), and field-programmable gate array (FPGA). ADC, analog-to-digital converter; TIS, trans-impedance amplifier; DAC, digital-to-analog converter. Image from: Science Advances
The team explained that retinomorphic systems that can see, recognize, and respond to real-time environmental information could extend the complexity and range of tasks that an exoskeleton robot can perform to better assist physically disabled people. However, the lack of ultrasensitive, reconfigurable, and large-scale integratable retinomorphic devices and advanced edge-processing algorithms makes it difficult to realize retinomorphic hardware.
According to the scientists, the strong broadband absorption, large photoresponse, controllable reconfigurability upon ion migration, and facile solution processability make organometallic halide perovskites able to ensure large and tunable photoresponsivity and provide the large-area integrating capability, so they are considered compelling materials for the construction of retinomorphic hardware systems.
A narrow bandgap (NBG) perovskite with strong ultraviolet–visible–near-infrared (UV-VIS-NIR) absorption, large on/off ratio, and reconfigurable photoresponsivity was used to fabricate the large-scale photodetector array on an amorphous Si thin-film transistor (a-Si TFT) panel.
The retinomorphic sensor array, using a one photodetector–one transistor geometry to resemble retinal circuit with broadband, ultrahigh, and reconfigurable photoresponsivities, executes both adaptive imaging with a contrast enhancement of ~620% under a dim-lit intensity of 10 microwatts per square centimeter and an instantaneous one-dimensional feature extraction algorithm to decompose the origin visual scenarios into parsimoniously encoded spatiotemporal information.
The team stated that this retinomorphic system grants embodied intelligence with adaptive imaging, in situ processing, and decision-making capabilities and promises enormous potential for autonomous robot applications. They say that this innovative retinomorphic computation system with instantaneous “see, recognize, and respond” capability promises enormous potential for future embodied intelligent vision applications.
