Before reading
- This is not a typical XAI method for every case. It required a highly sepcific dataset to form the explanation
- This work is an extention of NetDissect [1], which has been mentioned in previous note.
Overview
Since the use cases for datasets and models are identical to NetDissect, we will skip these two parts and go straight into explain the basic idea of this work.
Changes. In this section, we explain what are the basic idea of this work, and how is it different from NetDissect.
Result. What is the results they obtained, and how is it differnt to NetDissect.
Changes
NetDissect
This work is extended from a previous work called, NetDissect. NetDissect is to explain the neuron by searching for the concept $C$ that is most similar to the neoron’s activation maps. To determine which concept activate, this is the optimisation equation:
Where
DissectNet search the concepts to find the one can activate the specific neuron most. And, the neuron can be seen as a detector for that concept.
This work
In NetDissect, they are trying to find the one concept that can activate the neuron most. And, in this work, they believe the neuron is not only for detect ing single concept but for detecting a logical forms. In the image belwo, (d) is the single concept used in NetDissect. And this work is using the logical forms in (f). However, both of them use the smae IoU technique for finding the best matches.
Because this work is searching for a logical form (a set of concepts) instead of single concept, beam searching is used for finding the best composition of concepts.
So it’s not an extra loss function that forcing the neural network to learn semantic meaning. Instead of doing that, they proposed a method to search the Logical Form, which can be seen as the condition of occurence and absence of concepts. To be straightforward, this work is aiming to find logical forms of concepts for each neuron. Or we can say, the searching process is to “find which combimation (composition) has the highest with that neuron’s activation map (Mask for measuring IoU)”.
Natural Language Inference (NLI)
Task description. Since NLI is not used in NetDissect, we briefly describe this task here. Given premise and hypothesis sentences, the task of NLI is to determine whether the premise entails the hypothesis, contradicts it, or neither (neutral).
Example
Results
Image Classification Task
Natural Language Inference
Do interpretable neurons contribute to model accuracy?
A: Only for Image classification task.
Munipulate predictions
By understanding how the neoron is activated for classfication, we can manipulate the input to obtain desired result. It is like the feeling of counterfactual.
