Process Report

Table of Contents

• Introduction
• Input variables shared across all scripts
• parseSMreport.r
• Create a reporter_sample_template.txt file (horizontal layout)
• Create a sample-annotation.csv file (vertical layout)
• How to read/write a .GCT v1.3 file using R or Python
• plotRatioDistributions.r
• normalizeReporterRatios.r
• Running individual scripts from the command line
• Customizing individual scripts

Introduction

These SM report processing R scripts were architected with the following in mind:

1. Scripts could be run in multiple ways:
• Via a web browser, that navigates the Spectrum Mill filesystem.
• From the command line.
• Called from other R scripts
2. Individuals would modify a script for related uses.
3. Scripts would be used in pipelines other than Spectrum Mill.
4. Scripts would be run on either Windows or Unix.
5. Project-specific features would be added.

Input variables shared across all scripts

• Data Directories: Choose the directoreis which contain SM reports, not all of the directories which contributed to a report.
• Project: choices other than Generic are used to trigger customized features associated with a particular project:
  • CPTAC2 TCGA: specific means of parsing LC-MS/MS run names to extract sample reporter ion associations.
  • Matrisome: specific subsets of proteins are selected for separate handling in distribution plotting (ECM, Collagen, Fibrillar collagens) which are recognized by extra columns included in the report (Division, Category, Notes).
  • GO Categories Present: Additional columns present in the report are extracted and passed through by the parser.
• SM report type: Corresponds to the mode selected when generating a report from Protein/Peptide Summary. All reports are expected to have been generated with the run specific option enabled. The report type is used by the parser not only to determine what file name to retrieve from the data directory, but also to indicate what format to expect and trigger appropriate handling/presentation of the report's content.

parseSMreport.r

Input Options

• Reporter Ion label type: The type selected is used by the parser to specify how many data columns are expected to be present for each directory (the number of ratios should be the number of reporter ions - 1). However, TMT10 mean multi indicates 10 ratios are expected because when generating the SM report the control ion was designated as MeanMulti, leading to the mean intensity of multiple reporter ions was used as the denominator in ratios for each of the TMT10 mass labels.

Output Options

• .TXT - Simple tab-delimited output using either the original data column headers from P/P Summary or revised headers based on the content of a user provided reporter_sample_template.txt file (see below), if present in the data directory being processed.
• .GCT - (Gene Cluster Text) v1.3 is a tab-delimited text file format that is convenient for analysis of matrix-compatible datasets as it allows metadata about an experiment to be stored alongside the data from the experiment. In order to produce this output format the sample identifiers and metadata must be correlated with the reporter ion labels used in the experiment via a user provided sample-annotation.csv file (see below), present in the data directory being processed.

Create a reporter_sample_template.txt file (horizontal layout)

When the parseSMreport script is run to process a P/P Summary report it can update the column headers in the report to include meta information, like more specific sample names attached to each reporter ion. These names will also be propagated through to the ratio distribution plots, and reports following normalization.

1. line 1: reporter ion. The parser looks only for a substring in each cells text for the reporter ion (126, 127N, etc). The A and B are optional, and added only for user convenience.
2. line 2: sample name, free text.
3. line 3: directory name, directory the data was present in when the report was generated. This is necessary to match up the name and reporter ion, when more than 1 directory contributed to the report.

Special considerations;

• Do not use a colon character : in any cells of the report, because colon will later be inserted by the parser to denote the numerator : denominator involved in a ratio.
• The examples have the columns in mass order. However, you can put the columns in any order. The parsed report will come out with the same column order as the reporter_sample_template.txt.
• When saving reporter_sample_template.txt in Excel
  Save as type:
  Text (Tab delimited) (*.txt)
  not
  Unicode Text (*.txt)
  You can diagnose problems caused by saving as Unicode Text by the funky characters in the script output pane in the browser in the reporterIons field.
  numerator: 126 denominator: 127C reporterIons: ÿþ1

MeanMulti

Create a sample-annotation.csv file (vertical layout)

The requirements described below are, by design, harmonized between tools developed in the Broad Institute Proteomics Platform group including: Spectrum Mill and the downstream tools Protigy and Panoply.

In order for the parseSMreport script to correlate each reporter ion label with a sample name and metadata to produce a GCT v1.3 format output file you must provide a comma-delimited .csv file in the SM directory that contains the report that is parsed. While the filename may be prefixed at the user's discretion, the text file must be named with the suffix
     sample-annotation.csv
And be organized like the example below:

• Required columns are only: Sample.ID, Experiment, and Channel.
• Though not required for SM, the column: Type (Tumor, NAT, etc) is required downstream for Panoply use.
• Note that the sample-annotation file must not contain a row for the sample(s) used as a control ion (denominator in ratios). However, users are encouraged to include the isobaric label name and control ion channel in the filename of the sample-annotation file.
• The headers of metadata columns need to all be R-compatible. No spaces, no control characters like (), and can not begin with a number.
• Replicates: Sample.IDs must be unique and cannot have duplicates. If replicate samples are present, they must have unique Sample.IDs, but can be identified as replicates by using an additional column (named "Participant" for downstream Panoply use) with identical ids.
• The column Experiment (integer values) must be numbered in the exact same order as the data directories appear in the report (or Sample.ID and metadata will be mis-applied) , which is the same order as they were selected when creating the report in P/P Summary. Note that the SM data directory need not, but could be a column in the sample-annotation file.
• The column Channel should include the reporter ion label used in SM reports. Please note the following nuances: For TMT6, 10 SM reports: 126, 131, while for TMT11, 16, 18 SM reports: 126C, 131N
. The use of and N and C on every channel was implemented for clarity, and I regret not implementing the convention for the earliest reagents, in deference to the manufacturer's nomenclature, though considered it at the time. Now to ensure backwards compatibility the difference is what it is.

Sorting of the sample-annotation.csv file:

Example sample-annotation.csv file

TMT6-multimedian-2004-RedSox-reverse-the-curse-sample-annotation.csv

A hypothetical TMT6 experimental design contemplated at the ice cream social held across the river from Fenway Park in Cambridge, MA at the Broad Institute in late October of 2004. 3 TMT6 plexes with each of 9 samples run in duplicate for a P/P Summary report generated with MedianMulti as the control ion. Thus ratios were calculated using a denominator composed of the median of all 6 channels.

How to read/write a .GCT v1.3 file using R or Python

Now perhaps you are thinking:
     "Hmmm, this .GCT format is kind of cool in that the data and metadata are combined, but how am I going to read it in to my script so I can do my own analyses?"
Probably just install the GCT package, right? Ummm, well kinda sorta, but not really, at least not at the moment anyway. However, currently there are the following options:

Python

R

or

• SpectrumMill/millR/parseSMreport.r
• SpectrumMill/millR/normalizeReporterRatios.r

to use the cmapR functions: write_gct(), parse_gctx(), and subset_gct()

Full cmapR documentation is available at rdrr.io/github/cmap/cmapR/man/

plotRatioDistributions.r

normalizeReporterRatios.r

Running individual scripts from the command line

If you wish to run any of the individual R scripts from an MS-DOS command prompt do the following:

1. Install R on your local machine, download from: https://www.r-project.org/
2. Add Rscript.exe to the Windows system path on your local machine. Startup menu – control panels/system/advanced/environment variables/system variables/path
   c:\Program Files\R\R-3.2.2\bin\x64
3. Open an MS-DOS Command Window. (From the Windows Start menu, select All programs then Accessories then Command Prompt)
4. Change to the volume where the millR directory on your Spectrum Mill server is mounted.
   • If you have administrator privileges on the SM server and you have mounted Y: as E$(\\dunlop)
     Type 'Y:' then 'cd spectrumMill\millr' and the command prompt will read
     Y:\spectrumMill\millr>
   • If you do not have administrator privileges, but the millR directory has ben shared, then map a network drive:
     millR(\\dunlop)Y:
     Type 'Y:' and the command prompt will read
     Y:>
     Type 'dir' to verify that *.r files are present
5. Type lines like the following which run each script along with required parameters
• Rscript.exe parseSMreport.r proteinProteinCentricColumnsExport.1.ssv proteome TMT10 ..\msdataSM\Karl\test\
• Rscript.exe plotRatioDistributions.r proteinProteinCentricColumnsExport.1-ratio.txt 3 ..\msdataSM\Karl\test\
• Rscript.exe normalizeReporterRatios.r proteinProteinCentricColumnsExport.1-ratio.txt median ..\msdataSM\Karl\test\

Customizing individual scripts