Carissa Handrian’s scientific contributions

Machine learning models for predicting PET-hydrolyzing enzymes (PETases) are frequently evaluated solely on benchmark metrics, often overlooking whether their predictions are biologically sound. The InterPET framework introduces a curated benchmark and ablation study to evaluate sequence embeddings and graph architectures on both predictive performance and biological validity.

Dataset Curation

• The training set comprises 937 sequences, while the independent benchmark set contains 139 sequences.

• Data was sourced from PlasticDB, PAZy, PlasticEnz, and PEZY-miner.

• Duplicates and potential data leakage were rigorously removed using CD-HIT-2D filtering.

• Negative datasets consisted of enzymes experimentally confirmed to be active on non-PET plastics, rather than using randomly sampled non-enzymatic sequences.

Ablation Study Configurations

• Eight model configurations were tested across three sequence representations: ESM-2, ProtT5, and classical AAC/CTD descriptors.

• Tree-based classifiers, specifically XGBoost and Random Forest, were paired with these representations.

• Two GraphSAGE neural network variants were included to compare sequence-only adjacency against 3D structural contact-map edges.

Key Findings & Performance

• The ESM-2 + XGBoost model achieved the highest benchmark metrics, boasting an F1 score of 0.91, an AUC of 0.99, and an MCC of 0.90.

• Graph models utilizing 3D contact-map edges outperformed their sequence-only counterparts, indicating that explicit structural data provides additional predictive value.

• SHAP feature attribution on AAC/CTD models identified proline composition, solvent accessibility, and hydrophobicity as major biological determinants of PETase activity.

Biological Validation vs. Benchmark Accuracy

• In-silico alanine-scanning mutagenesis revealed a significant disconnect between raw accuracy and mechanistic trustworthiness.

• Despite scoring highest on standard metrics, the top-performing ESM-2 + XGBoost model recovered only an average of 0.5 out of 3 known catalytic-triad residues.

• The ProtT5 + Random Forest model demonstrated the strongest biological validity, consistently recovering catalytic-triad positions without showing off-target sensitivity to an unrelated hydrolase’s active site.

Relying on a single leaderboard metric can lead to selecting models that classify accurately but for biologically incorrect reasons. Future enzyme-function prediction pipelines should evaluate benchmark performance alongside interpretability and biological validity to ensure robust, mechanistically grounded utility.