Project: Interactive Pattern Specification and Search in Multivariate Time Series using Active Learning
Description
Multivariate time series data arise in many domains including healthcare, finance, biology, astronomy, and engineering. A common analytical task across these domains is pattern search: identifying occurrences of meaningful temporal behaviors within data. Given a pattern of interest, the goal is to retrieve other segments that are considered similar based on the user's analytical needs. Applications include detecting medical conditions from physiological signals, studying changes in the properties of astronomical objects, examining gene expression patterns to understand biological processes, identifying events in sensor streams, or finding recurring financial market behaviors.
However, what constitutes a pattern, and what makes two patterns similar, is highly subjective, task-dependent, and domain-dependent. Deep learning approaches can learn complex similarity relationships from data, but often lack interpretability and flexibility, making it unclear what notion of similarity has been learned, or how the system will behave under changing analytical goals. Adapting these models may require retraining, parameter tuning, or large amounts of labeled data, which limits their suitability for exploratory and interactive analysis where users expect immediate feedback and iterative refinement.
This project will explore the design of an interactive system for pattern search in multivariate time series along four main dimensions:
- Pattern specification (how users define patterns):
Support flexible and expressive ways of defining patterns, including sketch, example (single or multiple), and natural language. - Similarity and user control (what similar means):
Enable interpretable similarity measures that users can inspect and adjust, including control over which aspects of a pattern matter (e.g., shape, scale, timing, context). - Interaction and learning (how the system adapts):
Incorporate user feedback to iteratively refine search results through, e.g., relevance feedback, active learning, and explanations of retrieved matches. - Representation (what is being matched on):
May explore how different time series representations, such as time-domain, frequency-domain, or time-frequency-domain, can be integrated into the pattern specification and retrieval process.
Related Readings:
- F. Lekschas, B. Peterson, D. Haehn, E. Ma, N. Gehlenborg, and H. Pfister. Peax: Interactive visual pattern search in sequential data using unsupervised deep representation learning. Computer Graphics Forum, 39:167–179, 6 2020. doi: 10.1111/cgf.13971
- Y. Yu, D. Kruyff, J. Jiao, T. Becker, and M. Behrisch. Pseudo: Interactive pattern search in multivariate time series with locality-sensitive hashing and relevance feedback. IEEE Transactions on Visualization and Computer Graphics, 29:1–10, 1 2022. doi: 10.1109/TVCG.2022.3209431
Details
- Supervisor
-
Hrim Mehta
- Secondary supervisor
-
Fernando Paulovich
- Interested?
- Get in contact