Introduction

Imagine you're a data scientist developing a machine-learning model to predict wildfire risk. You aim to create a model that performs well and provides insights easily understood by stakeholders, such as firefighters and policymakers. This is where monosemanticity comes into play.

Monosemanticity: A Beginner's Guide

Monosemanticity focuses on creating features or representations with a clear, distinct meaning. Each feature should capture a specific concept or pattern in the data, making the model more interpretable and transparent.

Case Study: Wildfire Risk Assessment

We'll build two machine learning models to predict wildfire risk: one using monosemantic features and one using non-monosemantic features.

Step 1: Data Preprocessing

We begin by loading and preprocessing the datasets.

import pandas as pd

# Load datasets
exemption_notices = pd.read_csv('exemption_notices.csv')
working_forest_data = pd.read_csv('working_forest_data.csv')

# Merge datasets on 'OBJECTID'
merged_data = pd.merge(exemption_notices, working_forest_data, on='OBJECTID')

# Select relevant columns and handle missing values
features = merged_data[['REPORTD_AC', 'EX_YEAR', 'GIS_ACRES_x', 'WFMP_YEAR', 'WFN_YEAR', 'YARD', 'SILVI_1', 'SILVI_2']].fillna(0)
labels = merged_data['PLAN_STAT']

# Encode categorical features
features = pd.get_dummies(features, columns=['YARD', 'SILVI_1', 'SILVI_2'], drop_first=True)

Step 2: Monosemantic Model - Mapping Neuron Activation Patterns

Next, we employ dictionary learning to extract sparse and interpretable features from the preprocessed data.

from sklearn.decomposition import DictionaryLearning

# Apply dictionary learning
dict_learner = DictionaryLearning(n_components=5, random_state=42)
mapped_features = dict_learner.fit_transform(features)

# Convert to DataFrame for better readability
mapped_features_df = pd.DataFrame(mapped_features, columns=['Feature1', 'Feature2', 'Feature3', 'Feature4', 'Feature5'])

# Inspect the components
components = dict_learner.components_
components_df = pd.DataFrame(components, columns=features.columns)
print("Components (Basis Vectors):")
print(components_df)

Step 3: Training the Monosemantic Predictive Model

Using the mapped features, we construct and train a neural network model to predict wildfire risk.

from keras.models import Sequential
from keras.layers import Dense, Dropout

# Build neural network model
monosemantic_model = Sequential([
    Dense(64, input_dim=5, activation='relu'),
    Dropout(0.5),
    Dense(32, activation='relu'),
    Dropout(0.5),
    Dense(1, activation='linear')
])

# Compile model
monosemantic_model.compile(optimizer='adam', loss='mean_squared_error', metrics=['mean_absolute_error'])

# Train model
history = monosemantic_model.fit(mapped_features, labels, epochs=10, batch_size=32, validation_split=0.2)

Step 4: Training the Non-Monosemantic Predictive Model

Now, let's train a non-monosemantic model using the original features without dictionary learning.

# Build neural network model
non_monosemantic_model = Sequential([
    Dense(64, input_dim=features.shape[1], activation='relu'),
    Dropout(0.5),
    Dense(32, activation='relu'),
    Dropout(0.5),
    Dense(1, activation='linear')
])

# Compile model
non_monosemantic_model.compile(optimizer='adam', loss='mean_squared_error', metrics=['mean_absolute_error'])

# Train model
history_non_monosemantic = non_monosemantic_model.fit(features, labels, epochs=10, batch_size=32, validation_split=0.2)

Step 5: Generating Custom Responses and Decision Support

Monosemanticity allows generating meaningful and actionable responses based on model predictions.