Search results

(1 - 1 of 1)

Title

Learning with Contextual Feature Annotations

Creator

Wang, Juanyan

Date

2024

Description

Machine learning models have been increasingly used for prediction across a wide range of domains, including medical diagnosis, loan approvals...
Show moreMachine learning models have been increasingly used for prediction across a wide range of domains, including medical diagnosis, loan approvals, intrusion detection, autonomous driving, and many others. However, their development and application face several challenges. One major issue is that labeled data is often scarce; acquiring high-quality labeled data can require specialized knowledge, making the labeling process time-consuming and costly. Additionally, the internal mechanisms of these models are often highly complex and opaque, meaning their decision-making processes are not easily interpretable by humans. While some models can provide explanations for their decisions, these explanations still do not align well with human reasoning. To address these issues, I propose models that can learn more efficiently from additional domain knowledge and can mimic humans in decision-making by utilizing contextual feature annotations.First, I introduce an approach that allows humans to provide additional domain knowledge for learning in text classification tasks. Specifically, we ask human annotators to highlight segments of the text, called rationales, that serve as the evidence for their labeling decisions. In my approach, I define a new loss function that incorporates the rationale-based supervision, where a document containing rationales has a higher probability of being correctly classified than the same document with the rationales removed. The model leverages rationales in addition to the document labels during the training stage, and thus is able to learn effectively even with a limited number of labeled documents and also has the benefit of acting like humans.Second, I introduce a framework that mimics typical human decision-making behavior in predictive processes, where the model skims the full feature vector, decides which features are relevant for the case at hand, and makes a classification decision using only the selected features. The model utilizes class labels and additional contextual feature annotations to support the classification decisions during training. At test time, the model is able to perform context-aware feature selection and classification: providing both the classification decision and the human-understandable feature-level explanation for each specific sample.In the final chapter, I address the problem of imbalance in context-aware feature selection tasks by incorporating different costs for various types of feature selection errors into the training process. I propose three cost-sensitive strategies tailored to preferences in diverse real-world scenarios and also conduct an extensive study to analyze their behavior. Consequently, the model improves its ability to reduce balanced errors in both feature selection and classification tasks while offering greater flexibility to achieve the desired trade-off between False Positive and False Negative errors.
Show less