Contents

1 Overview of MACP package

Systematic mapping of multiprotein complexes formed by protein-protein interactions (PPIs) can enhance our knowledge and mechanistic basis of how proteins function in the cells. Co-fractionation coupled with mass spectrometry (CF-MS) is gaining momentum as a cost-effective strategy for charting protein assemblies under native conditions using high-resolution chromatography separation techniques (e.g., size-exclusion and ion-exchange) without the need for antibodies or tagging of individual proteins. CF-MS is initially developed for detecting native soluble human protein complexes from the cytosolic and nuclear extracts of cultured cells, and was later adapted to create the mitochondrial connectivity maps using the mitochondrial extracts of chemically cross-linked cultures of neuronal-like cells. To capture high-quality PPIs from CF-MS co-elution profile, we have developed a well standardized and fully automated CF-MS data analysis software toolkit, referred to as MACP (Macromolecular Assemblies from the Co-elution Profile) in an open-source R package, beginning with the processing of raw co-elution data to reconstruction of high-confidence PPI networks via supervised machine-learning.

Unlike existing software tools (EPIC, PrInCE), MACP facilitates CF-MS data analysis with flexible functions for data filtering and interaction scoring that can be tailored to user’s needs. In addition to the similarity scoring measures used in EPIC, MACP utilizes ten other co-elution profile similarity correlation metrics over the entire co-elution profile for scoring and predicting native macromolecular assemblies. MACP also offers an individual or an ensemble classifier to enhance the quality of predicted PPIs. Notably, MACP toolkit includes independent functions for creating the predictive model, and allowing users to handle imbalanced distribution of the training data set
(i.e., interacting and noninteracting pairs). Unlike aforesaid tools, MACP automatically estimates the prediction performance via a k-fold cross-validation using the training set. Lastly, due to its modular architecture, MACP: (1) Allows users to evaluate the model performance using external test sets, (2) Provides an optimized clustering procedure to improve the quality of predicted complexes using a grid optimization strategy, and (3) Offers functionalities that can be used to predict either mitochondrial or non-mitochondrial PPIs from various biological samples.

2 MACP computational core

For each CF-MS experimental dataset, the search output derived from database searching of co-elution proteomic experiment is summarized into a table matrix containing relative quantification of proteins across collected fractions. In the matrix table, rows correspond to identified proteins, each labelled with a unique name, and relative quantification of identified proteins for a corresponding fraction, are represented as columns. Following converting the search results into a compatible format, the MACP includes various data pre-processing to improve the prediction quality, including missing value imputation, data-noise reduction, and data normalization. Following data processing, MACP creates possible protein pairs for each elution experiment, followed by computing their corresponding co-elution scores via using up to 18 similarity metrics that emphasize different profile similarity aspects. For classification purposes, the literature-curated co-complex PPIs derived from a public database (e.g., CORUM) are then mapped onto a feature matrix (i.e., matrix containing similarity scores for potential protein pairs), which in turn will be used as an input set for the built-in machine learning algorithms to score potential proteins pairs. The probabilistic interaction network is further denoised and finally partitioned using parametrized unsupervised approaches to predict putative complexes.

3 Preparation

3.1 Installing MACP

Installation from CRAN:

install.packages('MACP')

To install the development version in R, run:

if(!requireNamespace("devtools", quietly = TRUE)) {
install.packages("devtools")
}
devtools::install_github("BabuLab-UofR/MACP")

Load the package and other libraries for data manipulation:

library(MACP)
library(dplyr)
library(tidyr)

3.2 Data preparation

The data files accepted by MACP are simple tables containing relative quantification of proteins across collected fractions. The rows correspond to identified proteins, each labelled with a unique name, and columns correspond to fractions. To demonstrate the use of MACP, we will use a demo co-elution data, derived from mitochondrial (mt) extracts of mouse brain culture, fractionated by (size-exclusion chromatography, SEC). An example of CF-MS input data,bundled with the MACP package, can be loaded with the following command:

# Loading the demo data
data(exampleData)
dim(exampleData)
## [1] 284  83
# Inspect the data 
glimpse(exampleData)
##  num [1:284, 1:83] 0 0 0 0 0 0 0 0 0 0 ...
##  - attr(*, "dimnames")=List of 2
##   ..$ : chr [1:284] "Q99L13" "P05202" "Q9CXJ4" "P55096" ...
##   ..$ : chr [1:83] "FR.1.1" "FR.2.1" "FR.3.1" "FR.4.1" ...

4 MACP computational workflow: step-by-step analysis

In the following section, we will describe the major steps in MACP, emphasizing the arguments adjusted by the user. This includes: (1) Data pre-processing, (2) Protein-protein interactions (PPIs) scoring, (3) Prediction and network denoising, as well as (4) Network-based prediction of protein complexes.

4.1 Pre-processing

  • data_filtering removes those proteins measured only in one fraction (i.e., one-hit-wonders) from further analysis. Besides the one-hit-wonders, MACP removes common contaminants (e.g., keratins) only for mouse and human organisms and frequent flyers (i.e., proteins observed in >80% of fractions).
data_p1 = data_filtering(exampleData)
# Inspect the number of retained proteins 
dim(data_p1)
## [1] 249  83
  • impute_MissingData imputes zero values for proteins either not detected by MS or not expressed in the cell by replacing missing values with an average of adjacent neighbors (i.e., fractions) values.
x <- data_p1
# Assign column 10 to zeros
x[,10] <- NA
data_p2 <- impute_MissingData(x)
  • scaling performs column- and row-wise normalization of protein co-elution profile matrix to correct for sample injection variation and fraction bias using the command line below:
data_p3 <- scaling(data_p1)
  • Optional keepMT removes all the non-mitochondrial proteins by mapping the co-eluted proteins from chromatography fractions to MitoCarta database using the ensuing command. Note that this function is only applicable to mouse or human organisms.
data_p3 <- keepMT(data_p3)
# Inspect the number of retained proteins 
dim(data_p3)
## [1] 211  83

4.2 Protein-protein interactions (PPIs) scoring

The next step is to compute similarity scores for each protein pair based on their co-elution profile, as proteins belonging to the same protein complex is expected to co-elute in the same or neighboring fractions, and thus will show similar elution profiles (i.e., high similarity score). Prior to calculating correlation similarity metrics from co-elution profiles, MACP discards proteins that do not occur in the same fraction across CF-MS experiments. By default, MACP considers 18 metrics (for details, see calculate_PPIscore documentation), to compute similarity of two protein elution profiles for all possible protein pairs in each CF-MS experiment using the ensuing command line. To further minimize the computational run time, MACP provides users with an option to choose an appropriate co-fractionation correlation score cut-off using the corr_cutoff argument (apex and pcc_p is not included), if argument corr_removal is set to TRUE.

The following command can be executed to compute features for pre-processed data:

set.seed(100)
scored_PPI <- calculate_PPIscore(data_p3,
                                corr_removal = FALSE)
## 
  |                                                                            
  |                                                                      |   0%
  |                                                                            
  |========                                                              |  11%
  |                                                                            
  |================                                                      |  22%
  |                                                                            
  |=======================                                               |  33%
  |                                                                            
  |===============================                                       |  44%
  |                                                                            
  |=======================================                               |  56%
  |                                                                            
  |===============================================                       |  67%
  |                                                                            
  |======================================================                |  78%
  |                                                                            
  |==============================================================        |  89%
  |                                                                            
  |======================================================================| 100%

4.3 Prediction and network denoising

4.3.1 Build reference data set

MACP uses a supervised machine learning algorithm to infer interactions; thus it requires a set of associated external attributes (i.e., class labels) for training purposes. To get class labels for training machine learning classifiers, MACP first retrieves the gold reference set from the CORUM database by getCPX function, followed by generating a reference set of true positive (i.e., intra-complex) and negative interactions (inter-complex) using the scored_PPI as input via generate_refInt function. Also, users can submit their list of reference complexes. The refcpx, bundled with the MACP package, is extracted from CORUM database.

  1. Load reference complexes:
data("refcpx")
  1. Generate class labels for interactions:
# separate the interaction pairs
PPI_pairs <-
  scored_PPI %>%
  separate(PPI, c("p1", "p2"), sep = "~") %>% select(p1,p2)

# Generate reference set of positive and negative interactions
class_labels <-
    generate_refInt(PPI_pairs[,c(1,2)],refcpx)
table(class_labels$label)
## 
## Negative Positive 
##      118      130

Optional Note that if the ratio of positive to negative of training set is imbalanced, users can perform under-sampling downSample technique to balance the ratio as the prediction works bests normally having 1:1 or 1:5 positive to negative protein pairs.

Optional Furthermore, if training set deemed insufficient, we urge users to provide MACP with suitable reference datasets defined based on biochemical approaches collected from literature or other public databases (e.g., IntAct, Reactome, GO) or one-to-one orthologous protein mapping between human and test species of interest using orthMappingCpx function provided in the MACP package as follow:

# for example to convert mouse complexes to human complexes
## load the mouse complexes
data("refcpx")
orth_mapping <- orthMappingCpx (refcpx,
  input_species = "mouse",
  output_species = "human",
  input_taxid = "10090",
  output_taxid = "9606")

4.3.2 Protein-protein interactions (PPIs) predction

Using input data scored_PPI and training data set class_labels, MACP creates a single composite probability score by combining each of the scored interactions from 18 different similarity measures either through an individual or ensemble of supervised machine-learning models (RF, GLM, SVM), including other base classifiers provided in the R caret package to enhance PPI prediction quality. MACP then evaluates the precision of the resulting networks through k-fold cross validation and the use of different performance measures such as Recall (Sensitivity), Specificity, Accuracy, Precision, F1-score, and Matthews correlation coefficient (MCC).

The corresponding formulae are as follows:

\[ Recall=Sensitivity=TPR=\frac{TP}{TP+FN} \]

\[ Specificity=1-FPR=\frac{TN}{TN+FP} \]

\[ Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \]

\[ Precision=\frac{TP}{TP+FP} \]

\[ F1=2 \text{*} \frac{Precision \text{*} Recall}{Precision + Recall} \]

\[ MCC=\frac{TP \text{*} TN - FP \text{*} FN}{\sqrt{(TP+FP)\text{*} (TP+FN)\text{*} (TN+FP)\text{*} (TN+FN)}} \]

The predPPI_ensemble function provided in the MACP package takes following parameters:

  • features A data frame with protein-protein interactions (PPIs) in the first column, and features to be passed to the classifier in the remaining columns. Note that, this data includes both unknown and known PPIs.
  • gd A data frame with gold_standard PPIs and class label indicating if such PPIs are positive or negative.
  • classifier Type of classifiers.
  • cv_fold Number of partitions for cross-validation.
  • plots Logical value, indicating whether to plot the performance of the predictive learning algorithm using k-fold cross-validation.
  • filename A character string, indicating the location and output pdf filename for performance plots. Defaults is temp() directory.

Predicting interactions with ensemble algorithm is simple as the following command:

set.seed(101)
predPPI_ensemble <- 
  ensemble_model(scored_PPI,
                  class_labels,
                  classifier = c("glm", "svmRadial", "ranger"),
                  cv_fold = 5,
                  plots = FALSE,
                  verboseIter = FALSE,
                  filename=file.path(tempdir(),"plots.pdf"))

# Subset predicted interactions 
pred_interactions <- predPPI_ensemble$predicted_interactions

When the plots argument set to TRUE, the ensemble_model function generates one pdf file containing three figures indicating the performance of the RF classier using k-fold cross-validation.

  • The first plot shows the Receiver Operating Characteristic (ROC) curve.

    Figure 1: ROC_Curve curve.

  • The second plot shows the Precision-Recall (PR) curve

    Figure 2: Precision-Recall (PR) curve.

  • The third plot shows the accuracy (ACC), F1-score ,positive predictive value (PPV),sensitivity (SE),and Matthews correlation coefficient (MCC) of ensemble classifier vs selected individual classifiers.

    Figure 3: Point plot.

4.3.2.1 Defining high-confidence PPIs based on ROC-curve

Once the composite score for each PPIs predicted from the machine learning classifiers are established, MACP uses the command line below to determine a probability-based cut-off based on auROC using the reference dataset for defining high-confidence PPIs.

roc_object <-
  inner_join(class_labels, pred_interactions, by ="PPI")
roc_object <-
  pROC::roc(roc_object$label,roc_object$Positive)
## Setting levels: control = Negative, case = Positive
## Setting direction: controls < cases
cutoff_roc <- 
  pROC::coords(roc_object, x="best", input="threshold", best.method="youden")

# Extract high-confidence network based on the cut-off reported from ROC curve
# Note that best-threshold can change depending on the input data
ThreshNet_PPI <- filter(pred_interactions, Positive >= cutoff_roc$threshold) 
dim(ThreshNet_PPI)
## [1] 5529    2

4.3.3 Network denoising

Before finalizing the co-elution network for discovering protein complexes, the MACP also allows further removal of erroneous edges in the form of false positive edges in the predicted networks using network topology via get_DenoisedNet function. Such noises in the interaction network have been shown to reduce the performance of complex prediction algorithms. To perform this step, get_DenoisedNet function requires the high-confidence predicted interactome ThreshNet_PPI. The get_DenoisedNet function then estimates two proteins’ connectivity in the given network based on the common neighborhood between two proteins. The rationale behind neighborhood similarity is that if two proteins expect to interact within the same protein complex, they expect to be perfectly associated through many short links in the interaction network. This function return connectivity measure ranges between 0 and 1. High score connectivity measures correspond to high confidence interactions (i.e., well connected proteins through many short paths), in contrast, low score connectivity measure represent low confidence interactions (i.e., little evidence based on network topology). Following computing denoising, this function automatically removes PPIs with score < 0.

To further denoise the high-confidence PPI network ThreshNet_PPI, we can run the following command:

ThreshNet_PPI <- 
  separate(ThreshNet_PPI, PPI, c("p1","p2"), sep = "~")
denoisedPPI <- get_DenoisedNet(ThreshNet_PPI)
dim(denoisedPPI)
## [1] 5115    3

Optional MACP also removes PPIs occurring between outer mt membrane (OMM) and matrix, between intermembrane space (IMS) and matrix, as well as between any subcellular mt compartment (except OMM) and cytosolic proteins as they deemed to be erroneous, and it is unlikely to be physically associated in a physiological context. Note that this function is only applicable to mouse or human organisms. The following command is used to remove the aforesaid PPIs:

finalPPI <- subcellular.mtPPI(denoisedPPI, organism = "mouse")
dim(finalPPI)
## [1] 4682    4

4.4 Network-based prediction of protein complexes

In the final step, MACP allows constructing a set of putative complexes from the predicted high-confidence protein interaction network (i.e., finalPPI) via two-stage clustering algorithms to identify clusters representing protein complexes.

First, each threshold network is partitioned via the ClusterONE clustering algorithm. The input data for this function must be in ppi_input_ClusterONE.txt format. Therefore, we previously saved the predicted high-confidence (finalPPI) in our system directory. Users can save the high-confidence network in their current directory using the following command:

finalPPI <- finalPPI[, -4] # drop the last column
# Set the directory to your current directory
setwd("user's current directory")
write.table(finalPPI, file = "ppi_input_ ClusterONE.txt", 
            quote = FALSE,
            col.names = F, row.names = F, sep = "\t")

Finally, we can run the get_clusters function to predict putative complexes via ClusterONE using the high-confidence network as input data. Then, for each densely connected region detected by ClusterONE, users can further apply the MCL algorithm to partition them into smaller clusters using the MCL_clustering function provided in the MACP package by using the putative threshold network (finalPPI) and predicted complexes from ClusterONE as inputs.

The get_clusters takes the following arguments:

  • csize An integer, the minimum size of the predicted complexes. Defaults to 2.
  • d A number, specifies the density of predicted complexes. Defaults to 0.3.
  • p An integer, specifies the penalty value for the inclusion of each node. Defaults to 2.
  • max_overlap A number, specifies the maximum allowed overlap between two clusters. Defaults to 0.8.
  • tpath A character string indicating the path to the project directory that contains the interaction data. Interactions data must be stored as ppi_input_ClusterONE.txt file and containing id1-id2-weight triplets.

The following command can be used to predict putative complexes using ClusterOne:

pred_cpx <- get_clusters(csize = 2, 
                         d = 0.3, p = 2,
                         max_overlap = 0.8,
                         tpath =file.path(system.file("extdata", 
                                                      package = "MACP")))
dim(pred_cpx)
## [1] 9 2

The MCL_clustering takes the following arguments:

  • hc_ppi High-confidence interactions data containing id1-id2-weight triplets.
  • predcpx A data.frame containing predicted complexes resulted from ClusterONE algorithm.
  • inflation MCL inflation parameter. Defaults to 9.
  • csize An integer, the minimum size of the predicted complexes. Defaults to 2.

The following command can be used to predict putative complexes using MCL:

pred_cpx_mcl <- 
  MCL_clustering(finalPPI, # High-confidence interactions
                 pred_cpx, # Putative complexes produced by clusterONE
                 inflation = 9, 
                 csize =2)
dim(pred_cpx_mcl)
## [1] 51  2

4.4.1 Clustering parameter optimization

As with many unsupervised machine learning methods, these algorithms include several tunable parameters that require optimization. Indeed, the performance of these algorithms is critically sensitive to the selection of these parameter values and varies depending on the input data. Therefore, the effect of such parameters needs to be optimized to obtain high-quality protein complexes.

To remedy this, MACP package includes function that select the best parameters for both clustering algorithms via grid search optimization. Briefly, for each parameter combination setting specified by the users, putative complexes are compared against an independent benchmark of known protein complexes (i.e., CORUM) via different external evaluation metrics (i.e., maximum matching ratio (MMR), accuracy (Acc), and overlap (O)) and those parameters yielding the highest final composite score (sum of O, Acc, and MMR) can then be used to predict complexes.

The cluster_tuning function performs the tuning process for ClusterONE algorithms and considers the density threshold d, penalty p and max_overlap value between clusters. At the same time, MCL_tuning function tunes the inflation parameter of MCL clustering. To avoid evaluation biases, we recommend to first reduce redundancy in the known reference complexes via EliminateCpxRedundance function. This function reduces redundancy in the reference complexes by first computing the overlap of two complexes via Jaccard index, followed by merging overlapping complexes with user-defined threshold.

To perform clusterONE optimization, we can use the following command:

# first load the reference complex
data("refcpx")
Clust_tuning_result <-
  cluster_tuning(refcpx, csize = 3, 
                d = c(0.3,0.4),
                p = c(2, 2.5),
                max_overlap = c(0.6,0.7),
                tpath =
                  file.path(system.file("extdata", package = "MACP")))

For instance, we observed run 8 achieved the maximum composite score, then we use them to predict our final predicted complexes via ClusterONE algorithm:

pred_cpx_optimized <- get_clusters(csize = 2, 
                         d = 0.4, p = 2.5,
                         max_overlap = 0.7,
                         tpath =file.path(system.file("extdata", 
                                                      package = "MACP")))
dim(pred_cpx_optimized)
## [1] 11  2

We can then tune the parameters for MLC clustering using the optimized clusters obtained from ClusterOne as input:

mcl_tuning_result <- 
  MCL_tuning(finalPPI,
             pred_cpx_optimized, 
             refcpx,
             inflation = c(6,8,9,10))

Finally, the inflation value (10) resulting in highest composite score can be used to drive the final set of complexes from the MCL algorithm.

final_clusters <- 
  MCL_clustering(finalPPI,
             pred_cpx_optimized, 
             inflation = 10, 
             csize = 2)
dim(final_clusters)
## [1] 44  2

4.4.2 GO and pathway enrichment analysis of protein complexes

To enable enrichment analysis for three GO domains (molecular function, cellular component and biological process), KEGG, REACTOME pathways, CORUM complexes, and HPA phenotype g:profiler is applied with an FDR p-value correction per each complex via the enrichmentCPX function.

For instance, the following command can be used to performs GO.BP functional enrichment analysis of predicted complexes pred_cpx, where id corresponds to the id of the predicted complexes:

enrich_result <- 
  enrichmentCPX(final_clusters,
                threshold = 0.05,
                sources = "GO:BP",
                p.corrction.method = "bonferroni",
                custom_bg = NULL,
                org = "mmusculus")
head(enrich_result[, c(1,4,12)], n = 4)
##   id      p_value                           term_name
## 1  2 9.628915e-09   carboxylic acid metabolic process
## 2  2 1.145073e-08           oxoacid metabolic process
## 3  2 1.240911e-08      organic acid metabolic process
## 4  2 4.049520e-07 purine nucleotide metabolic process

5 MACP computational workflow: one-step analysis

For users with less computational skills to run, we have created a one-step analysis function predPPI_MACP in MACP software using the command line described below (covers the entire data analysis pipeline from Step 4.1 to Step 4.4, excluding the cluster tuning steps), allowing user to upload the CF-MS data files and generating the graphical display of the results.

Prediction of interactomes and putative complexes via predPPI_MACP using default parameters is therefore as simple as the following command:

# Load the input data
data("exampleData")
# Known reference complexes
data("refcpx")
# Perform prediction
Prediction_output <-
  predPPI_MACP(exampleData,
  refcpx,
  keepMT =TRUE, # keep mt proteins
  subcellular_mtPPI = TRUE,
  tpath = tempdir())

The predicted high-confidence network can also be visualized in Cytoskape tool directly from R by using the RCy3 R package.

For example to visualize the high-confidence network HC_PPI, make sure to first launch Cytoscape and the run the following commands:

ig <- 
   igraph::graph_from_data_frame(Prediction_output$filteredPPI)
RCy3::createNetworkFromIgraph(ig,"myIgraph")

6 Session info

sessionInfo()
## R version 4.1.2 (2021-11-01)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 19045)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=C                          
## [2] LC_CTYPE=English_United States.1252   
## [3] LC_MONETARY=English_United States.1252
## [4] LC_NUMERIC=C                          
## [5] LC_TIME=English_United States.1252    
## system code page: 936
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] caret_6.0-93     lattice_0.20-45  ggplot2_3.4.0    entropy_1.3.1   
##  [5] minet_3.52.0     infotheo_1.2.0.1 tidyr_1.3.0      dplyr_1.1.0     
##  [9] MACP_0.1.0       BiocStyle_2.22.0
## 
## loaded via a namespace (and not attached):
##   [1] backports_1.4.1        Hmisc_4.7-2            plyr_1.8.8            
##   [4] igraph_1.3.5           lazyeval_0.2.2         splines_4.1.2         
##   [7] listenv_0.9.0          SnowballC_0.7.0        MCL_1.0               
##  [10] GenomeInfoDb_1.28.4    digest_0.6.31          foreach_1.5.2         
##  [13] htmltools_0.5.4        RcppDE_0.1.7           GO.db_3.13.0          
##  [16] arules_1.7-5           fansi_1.0.4            magrittr_2.0.3        
##  [19] checkmate_2.1.0        memoise_2.0.1          cluster_2.1.4         
##  [22] doParallel_1.0.17      recipes_1.0.4          globals_0.16.2        
##  [25] fastcluster_1.2.3      Biostrings_2.60.2      gower_1.0.1           
##  [28] matrixStats_0.63.0     hardhat_1.2.0          timechange_0.2.0      
##  [31] jpeg_0.1-10            colorspace_2.1-0       blob_1.2.3            
##  [34] xfun_0.37              crayon_1.5.2           RCurl_1.98-1.10       
##  [37] jsonlite_1.8.4         impute_1.68.0          survival_3.2-13       
##  [40] zoo_1.8-11             iterators_1.0.14       glue_1.6.2            
##  [43] gtable_0.3.1           ipred_0.9-13           zlibbioc_1.40.0       
##  [46] XVector_0.32.0         kernlab_0.9-32         future.apply_1.10.0   
##  [49] BiocGenerics_0.40.0    scales_1.2.1           DBI_1.1.3             
##  [52] Rcpp_1.0.10            viridisLite_0.4.1      htmlTable_2.4.1       
##  [55] foreign_0.8-84         bit_4.0.5              proxy_0.4-27          
##  [58] preprocessCore_1.56.0  Formula_1.2-4          stats4_4.1.2          
##  [61] lava_1.7.1             prodlim_2019.11.13     htmlwidgets_1.6.1     
##  [64] httr_1.4.4             RColorBrewer_1.1-3     pkgconfig_2.0.3       
##  [67] ptw_1.9-16             nnet_7.3-16            sass_0.4.5            
##  [70] deldir_1.0-6           utf8_1.2.3             dynamicTreeCut_1.63-1 
##  [73] tidyselect_1.2.0       rlang_1.0.6            reshape2_1.4.4        
##  [76] PRROC_1.3.1            AnnotationDbi_1.56.2   munsell_0.5.0         
##  [79] tools_4.1.2            cachem_1.0.6           cli_3.6.0             
##  [82] generics_0.1.3         RSQLite_2.2.20         ranger_0.14.1         
##  [85] evaluate_0.20          stringr_1.5.0          fastmap_1.1.0         
##  [88] yaml_2.3.7             ModelMetrics_1.2.2.2   knitr_1.42            
##  [91] bit64_4.0.5            purrr_1.0.1            KEGGREST_1.32.0       
##  [94] gprofiler2_0.2.1       future_1.31.0          nlme_3.1-153          
##  [97] compiler_4.1.2         rstudioapi_0.14        plotly_4.10.1         
## [100] png_0.1-8              e1071_1.7-13           tibble_3.1.8          
## [103] bslib_0.4.2            stringi_1.7.12         Matrix_1.5-3          
## [106] vctrs_0.5.2            pillar_1.8.1           lifecycle_1.0.3       
## [109] BiocManager_1.30.19    jquerylib_0.1.4        data.table_1.14.6     
## [112] bitops_1.0-7           R6_2.5.1               latticeExtra_0.6-30   
## [115] bookdown_0.32          gridExtra_2.3          lsa_0.73.3            
## [118] IRanges_2.28.0         parallelly_1.34.0      codetools_0.2-19      
## [121] philentropy_0.7.0      MASS_7.3-54            withr_2.5.0           
## [124] S4Vectors_0.32.3       GenomeInfoDbData_1.2.7 expm_0.999-7          
## [127] parallel_4.1.2         grid_4.1.2             rpart_4.1-15          
## [130] timeDate_4022.108      class_7.3-21           rmarkdown_2.20        
## [133] pROC_1.18.0            Biobase_2.54.0         WGCNA_1.72-1          
## [136] lubridate_1.9.1        base64enc_0.1-3        interp_1.1-3