publications

2022

1. SenSys ’22

   Telesonar: Robocall Alarm System by Detecting Echo Channel and Breath Timing

   Zhenyu Yan, Qun Song, Rui Tan, and 1 more author

   In The 20th ACM Conference on Embedded Networked Sensor Systems (2022)(Acceptance ratio: 52/208=25%)

   Abs PDF

   Massive fraudulent and phishing robocalls present threats to societies. The integration of artificial intelligence technologies, including dialogue and voice generation systems, renders the robocalls more deceptive. Existing countermeasures such as caller ID, call provenance, voiceprint, and fake voice detection have respective limitations and are heavyweight for end users’ smartphones. This paper studies detecting the acoustic echo channel on the remote end of a call based on the received voice. The positive detection result evidencing the physical setup of an audio system is indicative of a human caller. However, the acoustic echo cancellation mechanisms of most audio systems and the use of earphone/headset diminish echoes significantly. To address these issues, the proposed Telesonar transmits short chirps during the vulnerable time of echo cancellation, detects the tiny echo remnants from the received voice, and passively analyzes the timing of caller’s breath sounds to confirm a human caller. Extensive real experiments under a wide range of settings show that Telesonar correctly recognizes human callers with a rate of over 95%, while wrongly recognizing voice robots as human with a rate of 3.8%.

2. SenSys ’22

   BlastNet: Exploiting Duo-Blocks for Cross-Processor Real-Time DNN Inference

   Neiwen Ling, Xuan Huang, Zhihe Zhao, and 3 more authors

   In The 20th ACM Conference on Embedded Networked Sensor Systems (2022)(Acceptance ratio: 52/208=25%)
   Best Paper Award Candidate

   Abs PDF

   In recent years, Deep Neural Network (DNN) has been increasingly adopted by a wide range of time-critical applications running on edge platforms with heterogeneous multiprocessors. To meet the stringent timing requirements of these applications, heterogeneous CPU and GPU resources must be efficiently utilized for the inference of multiple DNN models. Such a cross-processor real-time DNN inference paradigm poses major challenges due to the inherent performance imbalance among different processors and the lack of real-time support for cross-processor inference from existing deep learning frameworks. In this work, we propose a new system named BlastNet that exploits duo-block - a new model inference abstraction to support highly efficient cross-processor real-time DNN inference. Each duo-block has a dual model structure, enabling efficient fine-grained inference alternatively across different processors. BlastNet employs a novel block-level Neural Architecture Search (NAS) technique to generate duo-blocks, which accounts for computing characteristics and communication overhead. The duo-blocks are optimized at design time and then dynamically scheduled to achieve high resource utilization of heterogeneous CPU and GPU at runtime. BlastNet is implemented on an indoor autonomous driving platform and three popular edge platforms. Extensive results show that BlastNetachieves 45.70% less deadline missing rate with a mere 1.63% of model accuracy loss.

3. SenSys ’22

   AutoMatch: Leveraging Traffic Camera to Improve Perception and Localization of Autonomous Vehicles

   Yuze He, Li Ma, Jiahe Cui, and 5 more authors

   In The 20th ACM Conference on Embedded Networked Sensor Systems (2022)(Acceptance ratio: 52/208=25%)

   Abs PDF

   Traffic camera is one of the most ubiquitous traffic facilities, providing high coverage of complex, accident-prone road sections such as intersections. This work leverages traffic camera to improve the perception and localization performance of autonomous vehicles at intersections. In particular, vehicles can expand their range of perception by matching the images captured by both the traffic cameras and on-vehicle cameras. Moreover, a traffic camera can match its images to an existing high-definition map (HD map) to derive centimeter-level location of the vehicles in its field of view. To this end, we propose AutoMatch - a novel system for real-time image registration, which is a key enabling technology for traffic camera-assisted perception and localization of autonomous vehicles. Our key idea is to leverage distinctive structures such as ground signs at intersections as the domain knowledge to facilitate image registration between traffic camera and HD map or vehicle. Moreover, by leveraging the strong structural characteristics of ground signs, AutoMatch can extract very few but precise keypoints for registration, which effectively reduces the communication/compute overhead. We implement AutoMatch on a testbed consisting of a self-built autonomous car, drones for surveying and mapping, and real traffic cameras. In addition, we collect two new multi-view traffic image datasets at intersections, which contain images from 220 real operational traffic cameras in 22 cities. Experimental results show that AutoMatch achieves pixel-level image registration accuracy within 88 milliseconds, and delivers an 11.65x improvement in accuracy and 1.42x speedup in compute time over state-of-the-art baselines.

4. SenSys ’22

   Indoor Smartphone SLAM with Learned Echoic Location Features

   Wenjie Luo, Qun Song, Zhenyu Yan, and 2 more authors

   In The 20th ACM Conference on Embedded Networked Sensor Systems (2022)(Acceptance ratio: 52/208=25%)

   Abs PDF

   Indoor self-localization is a highly demanded system function for smartphones. The current solutions based on inertial, radio frequency, and geomagnetic sensing may have degraded performance when their limiting factors take effect. In this paper, we present a new indoor simultaneous localization and mapping (SLAM) system that utilizes the smartphone’s built-in audio hardware and inertial measurement unit (IMU). Our system uses a smartphone’s loudspeaker to emit near-inaudible chirps and then the microphone to record the acoustic echoes from the indoor environment. Our profiling measurements show that the echoes carry location information with sub-meter granularity. To enable SLAM, we apply contrastive learning to construct an echoic location feature (ELF) extractor, such that the loop closures on the smartphone’s trajectory can be accurately detected from the associated ELF trace. The detection results effectively regulate the IMU-based trajectory reconstruction. Extensive experiments show that our ELF-based SLAM achieves median localization errors of 0.1m, 0.53m, and 0.4m in a living room, an office, and a shopping mall, and outperforms the Wi-Fi and geomagnetic SLAM systems.

5. MobiCom ’22

   VIPS: Real-Time Perception Fusion for Infrastructure-Assisted Autonomous Driving

   Shuyao Shi, Jiahe Cui, Zhehao Jiang, and 4 more authors

   In The 28th Annual International Conference on Mobile Computing and Networking (2022)(Acceptance ratio: 56/314=17.8%)
   Best Paper Award Runner-Up

   Abs HTML PDF

   Infrastructure-assisted autonomous driving is an emerging paradigm that expects to significantly improve the driving safety of autonomous vehicles. The key enabling technology for this vision is to fuse LiDAR results from the roadside infrastructure and the vehicle to improve the vehicle’s perception in real time. In this work, we propose VIPS, a novel lightweight system that can achieve decimeter-level and real-time (up to 100 ms) perception fusion between driving vehicles and roadside infrastructure. The key idea of VIPS is to exploit highly efficient matching of graph structures that encode objects’ lean representations as well as their relationships, such as locations, semantics, sizes, and spatial distribution. Moreover, by leveraging the tracked motion trajectories, VIPS can maintain the spatial and temporal consistency of the scene, which effectively mitigates the impact of asynchronous data frames and unpredictable communication/compute delays. We implement VIPS end-to-end based on a campus smart lamppost testbed. To evaluate the performance of VIPS under diverse situations, we also collect two new multi-view point cloud datasets using the smart lamppost testbed and an autonomous driving simulator, respectively. Experiment results show that VIPS can extend the vehicle’s perception range by 140% within 58 ms on average, and delivers a 4X improvement in perception fusion accuracy and 47X data transmission saving over existing approaches. A video demo of VIPS based on the lamppost dataset is available at https://youtu.be/zW4oi_EWOu0.

6. TOSN

   Physics-Directed Data Augmentation for Deep Model Transfer to Specific Sensor

   Wenjie Luo, Zhenyu Yan, Qun Song, and 1 more author

   ACM Transactions on Sensor Networks (2022)

   Abs HTML PDF

   Run-time domain shifts from the training phase caused by sensor characteristic variation incur performance drops of the deep learning-based sensing systems. To address this problem, existing transfer learning techniques require substantial target-domain data and incur high post-deployment overhead. Differently, we propose to exploit the first principle governing the domain shift to reduce the demand for target-domain data. Specifically, our proposed approach called PhyAug uses the first principle fitted with few labeled or unlabeled data pairs collected by the source sensor and the target sensor to transform the existing source-domain training data into the augmented target-domain data for calibrating the deep neural networks. In two audio sensing case studies of keyword spotting and automatic speech recognition, PhyAug recovers the recognition accuracy losses due to microphones’ characteristic variations by 37% to 72% with 5-second unlabeled data collected from the target microphones. In a case study of acoustics-based room recognition, PhyAug recovers the recognition accuracy loss caused by smartphone microphone variation by 33% to 80%. In the last case study of fisheye image recognition, PhyAug reduces the image recognition error due to the camera-induced distortions by 72%.

7. IPSN ’22

   BalanceFL: Addressing Class Imbalance in Long-Tail Federated Learning

   Xian Shuai, Yulin Shen, Siyang Jiang, and 3 more authors

   In The 21st ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) (2022)(Acceptance ratio: 38/126=30.2%)

   Abs HTML PDF

   Federated Learning (FL) is an emerging learning paradigm that enables the collaborative learning of different nodes without ex-posing the raw data. However, a critical challenge faced by the current federated learning algorithms in real-world applications is the long-tailed data distribution, i.e., in both local and global views, the numbers of classes are often highly imbalanced. This would lead to poor model accuracy on some rare but vital classes, e.g., those related to safety in health and autonomous driving applications. In this paper, we propose BalanceFL, a federated learning frame-work that can robustly learn both common and rare classes from a long-tailed real-world dataset, addressing both the global and local data imbalance at the same time. Specifically, instead of letting nodes upload a class-drifted model trained on imbalanced private data, we design a novel local update scheme that rectifies the class imbalance, forcing the local model to behave as if it were trained on ideal uniform distributed data. To evaluate the performance of BalanceFL, we first adapt two public datasets to the long-tailed federated learning setting, and then collect a real-life IMU dataset for action recognition, which includes more than 10,000 data sam-ples and naturally exhibits the global long tail effect and the local imbalance. On all of these three datasets, BalanceFL outperforms state-of-the-art federated learning approaches by a large margin.

8. IPSN ’22 Demo

   Demo Abstract: An Underwater Sonar-Based Drowning Detection System

   Lixing He, Haozheng Hou, Zhenyu Yan, and 1 more author

   In The 21st International Conference on Information Processing in Sensor Networks (Demo) (2022)

   Abs PDF

   Drowning is a major cause of unintentional deaths in swimming pools. Most swimming pools hire lifeguards for continuous surveillance, which is labor-intensive and hence unfeasible for small private pools. The existing unmanned surveillance solutions like camera array requires non-trivial installations, only work in certain conditions (e.g., with adequate ambient lighting), or raise privacy concerns. This demo presents SwimSonar, the first practical drowning detection system based on underwater sonar. SwimSonar employs an active ultrasonic sonar and features a novel sonar scanning strategy that balances the time and accuracy. Lastly, SwimSonar leverages a deep neural network for accurate drowning detection. Our experiments in real swimming pools show that the system achieves 88% classification accuracy with a scan time of 1.5 seconds.

9. IPSN ’22 Demo

   Demo Abstract: 3D Simultaneous Localization and Mapping with Power Network Electromagnetic Radiation

   Rongrong Wang, Zhenyu Yan, Rui Tan, and 1 more author

   In The 21st International Conference on Information Processing in Sensor Networks (Demo) (2022)

   Abs PDF

   Indoor localization by leveraging the existing residential instruments has been widely explored. Given the properties of temporal stability and spatial distinctness, the electromagnetic radiation (EMR) from the powerline network is a promising signal for location sensing. In this demo, we present a three-sensor setup to capture the powerline EMR signal from the three-dimensional (3D) space and formulate a new powerline EMR feature to implement the simultaneous localization and mapping (SLAM). Compared with the single sensor setup, our proposed approach can improve the localization accuracy to decimeter level.

2021

1. IPSN ’21

   PhyAug: Physics-Directed Data Augmentation for Deep Sensing Model Transfer in Cyber-Physical Systems

   Wenjie Luo, Zhenyu Yan, Qun Song, and 1 more author

   In The 20th International Conference on Information Processing in Sensor Networks (2021)(Acceptance ratio: 26/105=24.8%)
   Best Artifact Award Runner-Up

   Abs HTML PDF

   Run-time domain shifts from training-phase domains are common in sensing systems designed with deep learning. The shifts can be caused by sensor characteristic variations and/or discrepancies between the design-phase model and the actual model of the sensed physical process. To address these issues, existing transfer learning techniques require substantial target-domain data and thus incur high post-deployment overhead. This paper proposes to exploit the first principle governing the domain shift to reduce the demand on target-domain data. Specifically, our proposed approach called PhyAug, uses the first principle fitted with few labeled or unlabeled source/target-domain data pairs to transform the existing source-domain training data into augmented data for updating the deep neural networks. In two case studies of keyword spotting and DeepSpeech2-based automatic speech recognition, with 5-second unlabeled data collected from the target microphones, PhyAug recovers the recognition accuracy losses due to microphone characteristic variations by 37% to 72%. In a case study of seismic source localization with TDoA fingerprints, by exploiting the first principle of signal propagation in uneven media, PhyAug only requires 3% to 8% of labeled TDoA measurements required by the vanilla fingerprinting approach in achieving the same localization accuracy.

2. TMC

   Touch-to-Access Device Authentication for Indoor Smart Objects

   Zhenyu Yan, Qun Song, and Rui Tan

   IEEE Transactions on Mobile Computing (in press) (2021)

   Abs HTML PDF

   This paper presents TouchAuth, a new touch-to-access device authentication approach using induced body electric potentials (iBEPs) caused by the indoor ambient electric field that is mainly emitted from the building’s electrical network. The design of TouchAuth is based on the electrostatics of iBEP generation and a resulting property, i.e., the iBEPs at two close locations on the same human body are similar, whereas those from different human bodies are distinct. Extensive experiments verify the above property and show that TouchAuth achieves high-profile receiver operating characteristics in implementing the touch-to-access policy. Our experiments also show that a range of possible interfering sources including appliances’ electromagnetic emanations and noise injections into the power network do not affect the performance of TouchAuth. A key advantage of TouchAuth is that the iBEP sensing requires a simple analog-to-digital converter only, which is widely available on microcontrollers. Compared with the existing approaches including intra-body communication and physiological sensing, TouchAuth is a low-cost, faster, and easy-to-use approach for authorized users to access the smart objects found in indoor environments.

3. TOSN

   DeepMTD: Moving Target Defense for Deep Visual Sensing against Adversarial Examples

   Qun Song, Zhenyu Yan, and Rui Tan

   ACM Transactions on Sensor Networks (2021)

   Abs HTML PDF

   Deep learning-based visual sensing has achieved attractive accuracy but is shown vulnerable to adversarial attacks. Specifically, once the attackers obtain the deep model, they can construct adversarial examples to mislead the model to yield wrong classification results. Deployable adversarial examples such as small stickers pasted on the road signs and lanes have been shown effective in misleading advanced driver-assistance systems. Most existing countermeasures against adversarial examples build their security on the attackers’ ignorance of the defense mechanisms. Thus, they fall short of following Kerckhoffs’s principle and can be subverted once the attackers know the details of the defense. This article applies the strategy of moving target defense (MTD) to generate multiple new deep models after system deployment that will collaboratively detect and thwart adversarial examples. Our MTD design is based on the adversarial examples’ minor transferability across different models. The post-deployment of dynamically generated models significantly increase the bar of successful attacks. We also apply serial data fusion with early stopping to reduce the inference time by a factor of up to 5, as well as exploit hardware inference accelerators’ characteristics to strike better tradeoffs between inference time and power consumption. Evaluation based on three datasets including a road sign dataset and two GPU-equipped embedded computing boards shows the effectiveness and efficiency of our approach in counteracting the attack.

4. Book Chapter

   Clock Synchronization for Wide-Area Applications

   Zhenyu Yan, Yang Li, and Rui Tan

   (2021)
5. SenSys ’21 Demo

   Infrastructure-Free Smartphone Indoor Localization Using Room Acoustic Responses

   Dongfang Guo, Wenjie Luo, Chaojie Gu, and 4 more authors

   In The 19th ACM Conference on Embedded Networked Sensor Systems (Demo) (2021)

   Abs HTML PDF

   Smartphone indoor location awareness is increasingly demanded by a variety of mobile applications. The existing solutions for accurate smartphone indoor localization rely on additional devices or preinstalled infrastructure (e.g., dense WiFi access points, Bluetooth beacons). In this demo, we present EchoLoc, an infrastructure free smartphone indoor localization system using room acoustic response to a chirp emitted by the phone. EchoLoc consists of a mobile client for echo data collection and a cloud server hosting a deep neural network for location inference. EchoLoc achieves 95% accuracy in recognizing 101 locations in a large public indoor space and a median localization error of 0.5 m in a typical lab area. Demo video is available at https://youtu.be/5si0Cq6LzT4.

2020

1. PhD Thesis

   Exploiting Induced skin electric potential for body-area IoT system functions

   Zhenyu Yan

   (2020)

   HTML PDF

2019

1. MobiCom ’19

   Towards Touch-to-Access Device Authentication Using Induced Body Electric Potentials

   Zhenyu Yan, Qun Song, Rui Tan, and 2 more authors

   In The 25th Annual International Conference on Mobile Computing and Networking (2019)(Acceptance ratio: 55/290=18.9%)

   Abs HTML PDF

   This paper presents TouchAuth, a new touch-to-access device authentication approach using induced body electric potentials (iBEPs) caused by the indoor ambient electric field that is mainly emitted from the building’s electrical cabling. The design of TouchAuth is based on the electrostatics of iBEP generation and a resulting property, i.e., the iBEPs at two close locations on the same human body are similar, whereas those from different human bodies are distinct. Extensive experiments verify the above property and show that TouchAuth achieves high-profile receiver operating characteristics in implementing the touch-to-access policy. Our experiments also show that a range of possible interfering sources including appliances’ electromagnetic emanations and noise injections into the power network do not affect the performance of TouchAuth. A key advantage of TouchAuth is that the iBEP sensing requires a simple analog-to-digital converter only, which is widely available on microcontrollers. Compared with existing approaches including intra-body communication and physiological sensing, TouchAuth is a low-cost, lightweight, and convenient approach for authorized users to access the smart objects found in indoor environments.

2. SenSys ’19

   Moving Target Defense for Embedded Deep Visual Sensing against Adversarial Examples

   Qun Song, Zhenyu Yan, and Rui Tan

   In The 17th ACM Conference on Embedded Networked Sensor Systems (2019)(Acceptance ratio: 28/144=19%)

   Abs HTML PDF

   Deep learning-based visual sensing has achieved attractive accuracy but is shown vulnerable to adversarial example attacks. Specifically, once the attackers obtain the deep model, they can construct adversarial examples to mislead the model to yield wrong classification results. Deployable adversarial examples such as small stickers pasted on the road signs and lanes have been shown effective in misleading advanced driver-assistance systems. Many existing countermeasures against adversarial examples build their security on the attackers’ ignorance of the defense mechanisms. Thus, they fall short of following Kerckhoffs’s principle and can be subverted once the attackers know the details of the defense. This paper applies the strategy of moving target defense (MTD) to generate multiple new deep models after system deployment, that will collaboratively detect and thwart adversarial examples. Our MTD design is based on the adversarial examples’ minor transferability across different models. The post-deployment dynamically generated models significantly increase the bar of successful attacks. We also apply serial data fusion with early stopping to reduce the inference time by a factor of up to 5. Evaluation based on four datasets including a road sign dataset and two GPU-equipped Jetson embedded computing platforms shows the effectiveness of our approach.

2018

1. TMC

   Wearables Clock Synchronization Using Skin Electric Potentials

   Zhenyu Yan, Rui Tan, Yang Li, and 1 more author

   IEEE Transactions on Mobile Computing (2018)

   Abs HTML PDF

   Design of clock synchronization for networked nodes faces a fundamental trade-off between synchronization accuracy and universality of heterogeneous platforms, because a high synchronization accuracy generally requires platform-dependent hardware-level network packet timestamping. This paper presents TouchSync, a new indoor clock synchronization approach for wearables that achieves millisecond accuracy while preserving universality in that it uses standard system calls only, such as reading system clock, sampling sensors, and sending/receiving network messages. The design of TouchSync is driven by a key finding from our extensive measurements that the skin electric potentials (SEPs) induced by powerline radiation are salient, periodic, and synchronous on a same wearer and even across different wearers. TouchSync integrates the SEP signal into the universal principle of Network Time Protocol and solves an integer ambiguity problem by fusing the ambiguous results in multiple synchronization rounds to conclude an accurate clock offset between two synchronizing wearables. With our shared code, TouchSync can be readily integrated into any wearable applications. Extensive evaluation based on our Arduino and TinyOS implementations shows that TouchSync’s synchronization errors are below 3 and 7 milliseconds on the same wearer and between two wearers 10 kilometers apart, respectively.

2017

1. SenSys ’17

   Application-Layer Clock Synchronization for Wearables Using Skin Electric Potentials Induced by Powerline Radiation

   Zhenyu Yan, Yang Li, Rui Tan, and 1 more author

   In The 15th ACM Conference on Embedded Network Sensor Systems (2017)(Acceptance ratio: 26/151=17%)

   Abs HTML PDF

   Design of clock synchronization for networked nodes faces a fundamental trade-off between synchronization accuracy and universality for heterogeneous platforms, because a high synchronization accuracy generally requires platform-dependent hardware-level network packet timestamping. This paper presents TouchSync, a new indoor clock synchronization approach for wearables that achieves millisecond accuracy while preserving universality in that it uses standard system calls only, such as reading system clock, sampling sensors, and sending/receiving network messages. The design of TouchSync is driven by a key finding from our extensive measurements that the skin electric potentials (SEPs) induced by powerline radiation are salient, periodic, and synchronous on a same wearer and even across different wearers. TouchSync integrates the SEP signal into the universal principle of Network Time Protocol and solves an integer ambiguity problem by fusing the ambiguous results in multiple synchronization rounds to conclude an accurate clock offset between two synchronizing wearables. With our shared code, TouchSync can be readily integrated into any wearable applications. Extensive evaluation based on our Arduino and TinyOS implementations shows that TouchSync’s synchronization errors are below 3 and 7 milliseconds on the same wearer and between two wearers 10 kilometers apart, respectively.