Localization: re-discovering under-utilized hardware capabilities to create new applications and solutions

Abstract

Localization is a fundamental service that has enabled a plethora of applications and systems, such as enriching sensor data with location information in Wireless Sensor Networks and tracking device inputs in Human-Computer-Interface systems. The landscape of localization is still constantly evolving and there is currently no single localization method that satisfies all application scenarios and its intended needs. Instead of basing on the well known hardware setups and iterating on another incrementally better localization algorithm, this dissertation aims at creating unprecedented localization-enabled applications and system paradigms through rediscovering of and designing around under-utilized hardware capabilities. Many lessons are learned while the authors wander in the no man's land of utilizing these hardware capabilities to invent new applications/solutions, and hopefully will open up some new opportunities and pave some early roadways for future researchers/engineers interested to advance further. More specifically, we make the following discoveries and advancements: SmartLight: the first 3D digital sensor localization system. Since Spotlight was published in 2005 as the first 2D digital localization solution, the research community has long been looking for a 3D digital localization solution. Through digging into and utilizing the light refraction properties of lenses, we design and prototype ``SmartLight'', as the first 3D digital localization solution that elegantly solves the problems and provides superior robustness. This work won the best paper award at the world's top-tier research conference on sensor-networked systems: 2017 ACM Conference on Embedded Networked Sensor Systems. The system was also patented by US 11,051,270 B2. This is an important topic in the field because existing localization techniques are almost all analog-based, i.e., their accuracy heavily depends on the calibration of the model parameters and measurements of analog metrics such as angle of arrival (AOA) and received signal strength (RSS). It is well known that analog-based approaches usually require tedious calibration and suffer from performance degradation in noisy environments. ``SmartLight'' aims to shed light on the superior robustness and accuracy that digital approaches can offer. In a nutshell, ``SmartLight'' exploits light splitting geometry properties of convex lens to create an one-to-one mapping between a location and the set of orthogonal digital light signals receivable at that location. Advanced designs are also introduced to further improve the system accuracy and scalability beyond the hardware capability. Evaluation shows that it exhibited superior robustness against 1) environmental noise (e.g. sunlight, fluorescent lamps), 2) fluctuation and degradation of light source intensity, 3) receiver orientation, and 4) disturbance along light propagation paths (e.g. reflection and attenuation). BitLight: the first screen-sensor virtual touchscreen application. This is the first to imagine and prototype the conceptual application of sensor-display virtual touchscreen application ``BitLight''. It is only technically possible through utilizing and designing around the ultra-fine temporal division capabilities of DLP hardware. Intuitively it is motivated by the information-carrying property of light signals, and it is a novel paradigm that uses the rapid flashing of a digital light processing (DLP) projector to encode an imperceptible mask temporally that, when sensed by a photo-diode, uniquely specifies where the photo-diode is located on the projected image. This thus creates a ``free'' functionality of sensor localization during concurrent viewing of visual content. ``BitLight'' is inspired by the psycho-physical phenomenon that the human visual system (HVS) cannot resolve rapid temporal changes in optical signals, so redundant optical signals could be inserted for tracking with some or little compromise on the original human perceived visual content. ``BitLight'' is the first to devise a bit-level temporal encoding to display RGB colors while also embedding tracking signals in a digital fashion. Compared to traditional visible-light-communication (VLC) systems that use frame-level encoding techniques such as luminance changes and alpha channel, the bit-level encoding of ``BitLight'' makes better use of the ultra-fine temporal division capabilities of DLP projectors to embed a much higher tracking data throughput and thus achieve faster localization speed. With our current prototype hardware of a low-end micro-controller, cheap photo-diodes, and a commercial off-the-shelf DLP projector, evaluation results have demonstrated an average of only 9.5ms to localize the sensor, versus 200ms by a comparison test-bed that uses simple frame-level encoding. In summary, the innovation of the dissertation is primarily concentrated on finding new localization applications and solutions by rediscovering usages of under-utilized hardware capabilities. We design 1) the first-ever digital 3D light-based localization solution; 2) the first-ever screen-sensor virtual touchscreen application; The techniques proposed in the dissertation should have laid a pioneering and exploratory foundation for any future localization research on these new hardware applications and paradigms.