IDEAS Lab

IDEAS Lab

PI: Qi Zhu

Design Automation of Intelligent Systems Laboratory

Welcome to the IDEAS Lab at Northwestern University! Our research interests include design automation for intelligent cyber-physical systems (CPS) and Internet-of-Things (IoT) applications, safe, robust and data-efficient machine learning for embodied AI systems, cyber-physical security, and energy-efficient CPS. Our recent projects have been focusing on the cross-layer design, verification, and adaptation of learning-enabled cyber-physical systems (LE-CPS), particularly addressing the safety, robustness, security, energy, and data challenges in utilizing machine learning for CPS and in operating CPS within dynamic and uncertain environment. We work on applications in the domains of connected and autonomous vehicles, smart buildings and infrastructures, robotics, advanced manufacturing, IoT, etc.

Group News

  • [ACL 2025] Paper “Can LLMs Understand Unvoiced Speech? Exploring EMG-to-Text Conversion with LLMs” is accepted by the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) main conference. (5/2025)

  • [ICML 2025] Our co-authored paper “Directly Forecasting Belief for Reinforcement Learning with Delays” is accepted by the 42nd International Conference on Machine Learning (ICML). The work is in collaboration with University of Southampton and others. (5/2025).

  • [CVPR 2025] Paper “Split Adaptation for Pre-trained Vision Transformers” is accepted by the Conference on Computer Vision and Pattern Recognition (CVPR). (3/2025)

  • [Scientific Reports] Paper “Efficient and Assured Reinforcement Learning-based Building HVAC Control with Heterogeneous Expert-guided Training” is published by the Sceintific Reports. (3/2025)

  • [ICLR 2025] Papers “On Large Language Model Continual Unlearning” and “Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems” are accepted by the 13th International Conference on Learning Representations (ICLR). (1/2025)

  • [Neurips 2024] Papers “Semantic Feature Learning for Universal Unsupervised Cross-Domain Retrieval” and “Variational Delayed Policy Optimization” are accepted by the 38th Annual Conference on Neural Information Processing Systems (Neurips). (9/2024)

  • [IROS 2024] Papers “Graph Neural Network-Based Multi-Agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems” and “Kinematics-Aware Trajectory Generation and Prediction with Latent Stochastic Differential Modeling” are accepted by the International Conference on Intelligent Robots and Systems (IROS). (8/2024)

  • [Interspeech 2024] Paper “Missingness-resilient Video-enhanced Multimodal Disfluency Detection” is accepted by the Interspeech 2024 Conference. (6/2024)

  • [RA-L] Paper “Attrition-Aware Adaptation for Multi-Agent Patrolling” is accepted by the IEEE Robotics and Automation Letters. (5/2024)

  • [ICML 2024] Our co-authored paper “Boosting Reinforcement Learning with Strongly Delayed Feedback Through Auxiliary Short Delays” is accepted by the 41st International Conference on Machine Learning (ICML). The work is in collaboration with University of Southampton, University of Liverpool, and others. (5/2024)

  • ICLR 2024 Workshop on LLM Agents] Our paper “Empowering Autonomous Driving with Large Language Models: A Safety Perspective” is accepted by the ICLR 2024 Workshop on LLM Agents. (3/2024)

  • L4DC 2024] Our paper “State-Wise Safe Reinforcement Learning with Pixel Observations” is accepted by the 6th Learning for Dynamics and Control Conference (L4DC). (3/2024)

  • [ICASSP 2024] Our paper “Distribution-augmented Contrastive Reconstruction for Time-series Anomaly Detection”, in collaboration with GM, is accepted by the 49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). (12/2023)

  • [AAAI 2024] Our co-authored paper “REGLO: Provable Neural Network Repair for Global Robustness Properties” is accepted by the 38th Annual AAAI Conference on Artificial Intelligence (AAAI). The work is in collaboration with Boston University, University of Liverpool, and University of New Mexico. (12/2023)

  • [TCAD] Our paper “POLAR-Express: Efficient and Precise Formal Reachability Analysis of Neural-Network Controlled Systems”, is accepted by the IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD) for publication. The work is in collaboration with Boston University, University of Liverpool, and University of New Mexico. (12/2023)

  • [TMM] Our paper “Collaborative Multi-Agent Video Fast-Forwarding”, in collaboration with UC Riverside, is accepted by the IEEE Transactions on Multimedia (TMM). (12/2023)

  • [NSF grants] We received an NSF FM grant “Learning Foundation Models for Manufacturing Design Automation”; an NSF DESC grant on “FLEX: Building Future-proof Learning-Enabled Cyber-Physical Systems with Cross-Layer Extensible and Adaptive Design”, in collaboration with University of Pittsburgh; and an NSF Fuse grant on “R3AP: Retunable, Reconfigurable, Racetrack-Memory Acceleration Platform”, in collaboration with University of Pittsburgh and UCLA. (9/2023)

  • [MxD grant] We have started a project “Sensor Augmented Factory Environment (SAFE)” that is funded by the MxD (Manufacturing x Digital) Institute. The project is led by Boeing, in collaboration with Northwestern. (8/2023)

  • [MM 2023] Our paper “Effect of Attention and Self-Supervised Speech Embeddings on Non-Semantic Speech Tasks” is accepted by the ACM Multimedia (MM) as part of the Grand Challenges Track (8/2023)

  • [ICCV 2023] Our paper “Semi-supervised Semantics-guided Adversarial Training for Robust Trajectory Prediction” is accepted by the International Conference on Computer Vision (ICCV). The arXiv version can be found here. (7/2023)

  • [IROS 2023] Our papers “Learning Representation for Anomaly Detection of Vehicle Trajectories” (arXiv version here) and “Safety-Assured Speculative Planning with Adaptive Prediction” (arXiv version here) are accepted by the International Conference on Intelligent Robots and Systems (IROS). (7/2023)

  • [DAC 2023] We presented an invited paper “Waving the Double-Edged Sword: Building Resilient CAVs with Edge and Cloud Computing” at the 60th Design Automation Conference (DAC). The paper is based on joint work with University of California, Irvine. (7/2023)</li>

  • [ICML 2023] Our paper “Enforcing Hard Constraints with Soft Barriers: Safe-driven Reinforcement Learning in Unknown Stochastic Environments” is accepted by the 40th International Conference on Machine Learning (ICML). The arXiv version can be found here. (4/2023)

  • [ICASSP 2023] Our paper “Efficient Stuttering Event Detection using Siamese Networks” is accepted by the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). We also participated in the Signal Processing Grand Challenge “e-Prevention: Person Identification and Relapse Detection from Continuous Recordings of Biosignals” and earned the Third Place in Track 1 (Personal Identification). (2/2023)

  • [ICLR 2023] Our paper “DEJA VU: Continual Model Generalization For Unseen Domains” is accepted by the 11th International Conference on Learning Representations (ICLR). The arXiv version is here. (1/2023)

  • [DATE 2023] Our co-authored paper “A Safety-guaranteed Framework for Neural-network-based Planners in Connected Vehicles under Communication Disturbance” is accepted at the ACM/IEEE Design, Automation and Test in Europe Conference (DATE). (1/2023)

  • [MEMOCODE 2023] Prof. Zhu serves as a Program Committee Co-Chair for the 21st ACM/IEEE International Symposium on Formal Methods and Models for System Design (MEMOCODE). Please consider submit your work here! (1/2023)

  • [ASP-DAC 2023] Our paper “Safety-Driven Interactive Planning for Neural Network-Based Lane Changing” was presented at the 28th ACM/IEEE Asia and South Pacific Design Automation Conference (ASP-DAC) and nominated as a Best Paper Candidate. Another paper “Mixed-Traffic Intersection Management Utilizing Connected and Autonomous Vehicles as Traffic Regulators” we co-authored was also presented at the conference. (1/2023)