publications

2024

1. 

   AutoVP: An Automated Visual Prompting Framework and Benchmark

   Hsi-Ai Tsao*, Lei Hsiung*, Pin-Yu Chen, Sijia Liu, and Tsung-Yi Ho (* Equal Contribution)

   In Proceedings of the Twelfth International Conference on Learning Representations (ICLR) (2024)

   Abs arXiv Code

   Visual prompting (VP) is an emerging parameter-efficient fine-tuning approach to adapting pre-trained vision models to solve various downstream image-classification tasks. However, there has hitherto been little systematic study of the design space of VP and no clear benchmark for evaluating its performance. To bridge this gap, we propose AutoVP, an end-to-end expandable framework for automating VP design choices, along with 12 downstream image-classification tasks that can serve as a holistic VP-performance benchmark. Our design space covers 1) the joint optimization of the prompts; 2) the selection of pre-trained models, including image classifiers and text-image encoders; and 3) model output mapping strategies, including nonparametric and trainable label mapping. Our extensive experimental results show that AutoVP outperforms the best-known current VP methods by a substantial margin, having up to 6.7% improvement in accuracy; and attains a maximum performance increase of 27.5% compared to linear-probing (LP) baseline. AutoVP thus makes a two-fold contribution: serving both as an efficient tool for hyperparameter tuning on VP design choices, and as a comprehensive benchmark that can reasonably be expected to accelerate VP’s development.

2023

1. 

   Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbations

   Lei Hsiung, Yun-Yun Tsai, Pin-Yu Chen, and Tsung-Yi Ho

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

   Abs arXiv Code Demo

   Model robustness against adversarial examples of single perturbation type such as the \(\ell_p\)-norm has been widely studied, yet its generalization to more realistic scenarios involving multiple semantic perturbations and their composition remains largely unexplored. In this paper, we first propose a novel method for generating composite adversarial examples. Our method can find the optimal attack composition by utilizing component-wise projected gradient descent and automatic attack-order scheduling. We then propose generalized adversarial training (GAT) to extend model robustness from \(\ell_p\)-ball to composite semantic perturbations, such as the combination of Hue, Saturation, Brightness, Contrast, and Rotation. Results obtained using ImageNet and CIFAR-10 datasets indicate that GAT can be robust not only to all the tested types of a single attack, but also to any combination of such attacks. GAT also outperforms baseline \(\ell_∞\)-norm bounded adversarial training approaches by a significant margin.

2. 

   NCTV: Neural Clamping Toolkit and Visualization for Neural Network Calibration

   Lei Hsiung, Yung-Chen Tang, Pin-Yu Chen, and Tsung-Yi Ho

   In Proceedings of the 37th AAAI Conference on Artificial Intelligence Demo Track, AAAI (2023)

   Abs arXiv Code Demo

   With the advancement of deep learning technology, neural networks have demonstrated their excellent ability to provide accurate predictions in many tasks. However, a lack of consideration for neural network calibration will not gain trust from humans, even for high-accuracy models. In this regard, the gap between the confidence of the model’s predictions and the actual correctness likelihood must be bridged to derive a well-calibrated model. In this paper, we introduce the Neural Clamping Toolkit, the first open-source framework designed to help developers employ state-of-the-art model-agnostic calibrated models. Furthermore, we provide animations and interactive sections in the demonstration to familiarize researchers with calibration in neural networks. A Colab tutorial on utilizing our toolkit is also introduced.

3. 

   NeuralFuse: Learning to Improve the Accuracy of Access-Limited Neural Network Inference in Low-Voltage Regimes

   Hao-Lun Sun, Lei Hsiung, Nandhini Chandramoorthy, Pin-Yu Chen, and Tsung-Yi Ho

   Under Review. Preliminary version in NeurIPS workshop. (2023)

   Abs arXiv Code

   Deep neural networks (DNNs) have become ubiquitous in machine learning, but their energy consumption remains a notable issue. Lowering the supply voltage is an effective strategy for reducing energy consumption. However, aggressively scaling down the supply voltage can lead to accuracy degradation due to random bit flips in static random access memory (SRAM) where model parameters are stored. To address this challenge, we introduce NeuralFuse, a novel add-on module that addresses the accuracy-energy tradeoff in low-voltage regimes by learning input transformations to generate error-resistant data representations. NeuralFuse protects DNN accuracy in both nominal and low-voltage scenarios. Moreover, NeuralFuse is easy to implement and can be readily applied to DNNs with limited access, such as non-configurable hardware or remote access to cloud-based APIs. Experimental results demonstrate that, at a 1% bit error rate, NeuralFuse can reduce SRAM memory access energy by up to 24% while improving accuracy by up to 57%. To the best of our knowledge, this is the first model-agnostic approach (i.e., no model retraining) to address low-voltage-induced bit errors.

4. 

   Reducing surgical site infections after spine surgery: the optimal amount of normal saline for intra-wound irrigation

   Wei Hsiung, Yu-Cheng Yao, Hsi-Hsien Lin, Shih-Tien Wang, Lei Hsiung, Kuan-Jung Chen, Ming-Chau Chang, and Po-Hsin Chou

   The Spine Journal (2023)

   SpineWeek 2023 Best Paper Finalists.

   TL;DR HTML

   This study investigated the impact of intraoperative wound irrigation with saline solution on surgical site infections (SSI) following lumbar surgery. It found that insufficient irrigation and diabetes were risk factors for SSI, recommending at least 1,400 mL/hour of saline irrigation to reduce SSI in lumbar spine surgery.

2022

1. 

   A Lab-Based Investigation of Reaction Time and Reading Performance Using Different In-Vehicle Reading Interfaces during Self-Driving

   Lei Hsiung, Yung-Ju Chang, Wei-Ko Li, Tsung-Yi Ho, and Shan-Hung Wu

   In Proceedings of the 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI (2022)

   Abs PDF

   The demand for autonomous vehicles (AVs) is rapidly growing these years. As AVs have a potential to free drivers’ cognitive resources from driving to other tasks, reading is one of the common activities users conduct in travel multitasking. Nevertheless, ways to supporting reading in AVs have been little explored. To fill this gap, we explored the design of an in-vehicle reader on a windshield in AVs along three dimensions: dynamics, position, and text segmentation. We conducted two in-lab within-subject experiments to examine the eight kinds of in-car reading modalities that represented the combinations of the three dimensions in terms of drivers’ reaction time and reading comprehension. Our results show a case where an adaptive positioning would be particularly beneficial for supporting reading in AVs. And our general suggestion is to use a static reading zone presented on-sky and in sentences because it leads to faster reaction and better reading comprehension.

2. 

   CARBEN: Composite Adversarial Robustness Benchmark

   Lei Hsiung, Yun-Yun Tsai, Pin-Yu Chen, and Tsung-Yi Ho

   In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence Demo Track, IJCAI (2022)

   Abs arXiv Demo Poster

   Prior literature on adversarial attack methods has mainly focused on attacking with and defending against a single threat model, e.g., perturbations bounded in Lp ball. However, multiple threat models can be combined into composite perturbations. One such approach, composite adversarial attack (CAA), not only expands the perturbable space of the image, but also may be overlooked by current modes of robustness evaluation. This paper demonstrates how CAA’s attack order affects the resulting image, and provides real-time inferences of different models, which will facilitate users’ configuration of the parameters of the attack level and their rapid evaluation of model prediction. A leaderboard to benchmark adversarial robustness against CAA is also introduced.