Conditional Adaption Approach¶
import numpy as np
import pytorch_lightning as pl
import rul_datasets
import rul_adapt
The conditional adaption approaches are variants of the MMD and DANN approaches. In addition to applying their adaption loss to the whole data, they also apply it separately to pre-defined subsets. Each subset represents a different condition and is defined by a fuzzy set with a rectangular membership function.
In the original paper, three subsets are defined based on the RUL values of the source domain. The first fuzzy set contains healthy data which has a RUL value smaller than the median. The second fuzzy set contains degrading data with a RUL value between the 25th and 75th percentile. The last fuzzy set contains faulty data with a RUL value larger than the median. These sets are overlapping by design to take the uncertainty of sample membership into account.
We will extract the fuzzy sets' boundaries from the training split of CMAPSS FD003.
fd3 = rul_datasets.CmapssReader(3)
_, targets = fd3.load_split("dev")
targets = np.concatenate(targets) # combine all runs
median = np.median(targets)
lower_quart = np.quantile(targets, 0.25)
upper_quart = np.quantile(targets, 0.75)
fuzzy_sets = [(median, fd3.max_rul), (lower_quart, upper_quart), (0.0, median)]
print(fuzzy_sets)
[(110.0, 125), (55.0, 125.0), (0.0, 110.0)]
Now, we can use these fuzzy sets to adapt FD003 to FD001 with conditional MMD. First, we define the adaption data module.
fd1 = rul_datasets.CmapssReader(1, percent_broken=0.8)
dm = rul_datasets.DomainAdaptionDataModule(
rul_datasets.RulDataModule(fd3, batch_size=128),
rul_datasets.RulDataModule(fd1, batch_size=128)
)
Next, we instantiate a conditional MMD approach with a simple CNN feature extractor.
feature_extractor = rul_adapt.model.CnnExtractor(14, [16, 32], 30, fc_units=64)
regressor = rul_adapt.model.FullyConnectedHead(
64, [1], act_func_on_last_layer=False
)
approach = rul_adapt.approach.ConditionalMmdApproach(
lr=0.001,
mmd_factor=1.0,
num_mmd_kernels=5,
dynamic_adaptive_factor=0.5,
fuzzy_sets=fuzzy_sets
)
approach.set_model(feature_extractor, regressor)
We train the approach for 10 epochs and evaluate it on the test split of FD001.
trainer = pl.Trainer(max_epochs=10)
trainer.fit(approach, dm)
trainer.test(approach, dm)
We can also use the conditional DANN approach. This will use the same feature extractor and regressor as before and an additional domain discriminator. The discriminator network will be copied for each fuzzy set.
feature_extractor = rul_adapt.model.CnnExtractor(14, [16, 32], 30, fc_units=64)
regressor = rul_adapt.model.FullyConnectedHead(
64, [1], act_func_on_last_layer=False
)
domain_disc = rul_adapt.model.FullyConnectedHead(
64, [1], act_func_on_last_layer=False
)
approach = rul_adapt.approach.ConditionalDannApproach(
lr=0.001,
dann_factor=1.0,
dynamic_adaptive_factor=0.5,
fuzzy_sets=fuzzy_sets
)
approach.set_model(feature_extractor, regressor, domain_disc)
Again, we will train for 10 epochs and evaluate on the test split of FD001.
trainer = pl.Trainer(max_epochs=10)
trainer.fit(approach, dm)
trainer.test(approach, dm)
GPU available: False, used: False TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs HPU available: False, using: 0 HPUs | Name | Type | Params -------------------------------------------------------------------- 0 | train_source_loss | MeanAbsoluteError | 0 1 | mmd_loss | MaximumMeanDiscrepancyLoss | 0 2 | conditional_mmd_loss | ConditionalAdaptionLoss | 0 3 | evaluator | AdaptionEvaluator | 0 4 | _feature_extractor | CnnExtractor | 55.6 K 5 | _regressor | FullyConnectedHead | 65 -------------------------------------------------------------------- 55.6 K Trainable params 0 Non-trainable params 55.6 K Total params 0.223 Total estimated model params size (MB)
Sanity Checking: 0it [00:00, ?it/s]
Training: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
Validation: 0it [00:00, ?it/s]
`Trainer.fit` stopped: `max_epochs=10` reached.
Testing: 0it [00:00, ?it/s]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Test metric DataLoader 0 DataLoader 1
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
test/source/rmse 15.53573226928711
test/source/score 615.2120361328125
test/target/rmse 23.868637084960938
test/target/score 2933.725830078125
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[{'test/source/rmse/dataloader_idx_0': 15.53573226928711,
'test/source/score/dataloader_idx_0': 615.2120361328125},
{'test/target/rmse/dataloader_idx_1': 23.868637084960938,
'test/target/score/dataloader_idx_1': 2933.725830078125}]