REPRINT analysis #298

tobiasko · 2021-12-10T13:35:21Z

tobiasko
Dec 10, 2021

Dear Philosopher developers,

we are trying to setup a std. data analysis pipeline for AP-MS and proximity labeling (BioID, APEX, ...) experiments using Philosopher. The pipeline runs on Linux and follows

https://github.com/Nesvilab/philosopher/wiki/REPRINT-Analysis

but instead of using Comet, we decided for MSFragger as a search engine. We are also running IonQuant on top in order to have the option to forward labelfree quant results (int).

Questions:

Is there anything that one needs to keep in mind when moving from spc data to int data with respect to philosopher parameter choices? We would like to pass the reprint.spc.tsv and reprint.int.tsv to the REPRINT server for interaction scoring or use a local SAINTexpress instance. I am for instance unsure if IonQuant normalization should be applied or not, or if mbr should be restricted somehow.

Best,
Tobi

Answered by anesvi

Dec 10, 2021

For AP-MS, intensities from IonQuant, it is better to use topN approach. But there are two questions: 1. if you are running philosopher pipeline, how do you execute IonQuant? In FragPIpe, IonQuant rewrites reprint.tsv tables generated by philosopher. 2. Fengchao, what intensities are you writing in reprint.int.tsv? Ideally, we want topN. Do we alays calculate topN now, even if MaxLFQ is checked? Alexey From: Tobias Kockmann ***@***.***> Sent: Friday, December 10, 2021 8:36 AM To: Nesvilab/philosopher ***@***.***> Cc: Subscribed ***@***.***> Subject: [Nesvilab/philosopher] REPRINT analysis (Discussion #298) External Email - Use Caution Dear Philosopher developers, we are trying to setup a …

View full answer

anesvi · 2021-12-10T14:52:15Z

anesvi
Dec 10, 2021
Maintainer

For AP-MS, intensities from IonQuant, it is better to use topN approach. But there are two questions: 1. if you are running philosopher pipeline, how do you execute IonQuant? In FragPIpe, IonQuant rewrites reprint.tsv tables generated by philosopher. 2. Fengchao, what intensities are you writing in reprint.int.tsv? Ideally, we want topN. Do we alays calculate topN now, even if MaxLFQ is checked? Alexey From: Tobias Kockmann ***@***.***> Sent: Friday, December 10, 2021 8:36 AM To: Nesvilab/philosopher ***@***.***> Cc: Subscribed ***@***.***> Subject: [Nesvilab/philosopher] REPRINT analysis (Discussion #298) External Email - Use Caution Dear Philosopher developers, we are trying to setup a std. data analysis pipeline for AP-MS and proximity labeling (BioID, APEX, ...) experiments using Philosopher. The pipeline runs on Linux and follows https://github.com/Nesvilab/philosopher/wiki/REPRINT-Analysis but instead of using Comet, we decided for MSFragger as a search engine. We are also running IonQuant on top in order to have the option to forward labelfree quant results (int). Questions: Is there anything that one needs to keep in mind when moving from spc data to int data with respect to philosopher parameter choices? We would like to pass the reprint.spc.tsv and reprint.int.tsv to the REPRINT server for interaction scoring or use a local SAINTexpress instance. I am for instance unsure if IonQuant normalization should be applied or not, or if mbr should be restricted somehow. Best, Tobi — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#298>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIIMM62BBLNIQ4J2X77XQGLUQH62LANCNFSM5JZE5SKQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

1 reply

fcyu Dec 10, 2021
Maintainer

Fengchao, what intensities are you writing in reprint.int.tsv? Ideally, we want topN. Do we alays calculate topN now, even if MaxLFQ is checked?

If using the latest IonQuant, the intensity in reprint.int.tsv is always from topN. Currently, we write both topN and MaxLFQ if MaxLFQ is checked, and only topN if MaxLFQ is unchecked.

Best,

Fengchao

anesvi · 2021-12-10T15:04:06Z

anesvi
Dec 10, 2021
Maintainer

Ok, that’s good. And with top-N, I think I usually use N=6 for AP-MS. This being said, I often get better results with spectral counts still when looking at SAINT results. Or at least different from when using the intensity data from the same run. The two seem to be complementary, with my personal preference for spectral count-based analysis. From: Fengchao ***@***.***> Sent: Friday, December 10, 2021 9:56 AM To: Nesvilab/philosopher ***@***.***> Cc: Nesvizhskii, Alexey ***@***.***>; Comment ***@***.***> Subject: Re: [Nesvilab/philosopher] REPRINT analysis (Discussion #298) External Email - Use Caution 1. Fengchao, what intensities are you writing in reprint.int.tsv? Ideally, we want topN. Do we alays calculate topN now, even if MaxLFQ is checked? If using the latest IonQuant, the intensity in reprint.int.tsv is always from topN. Currently, we write both topN and MaxLFQ if MaxLFQ is checked, and only topN if MaxLFQ is unchecked. Best, Fengchao — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#298 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIIMM64WLIZWL7KPEH4E2QLUQIIJJANCNFSM5JZE5SKQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

1 reply

tobiasko Dec 17, 2021
Author

@anesvi I also just cross compared the SAINTexpress results for the same raw files using spc and int-based inputs (reprint files). They differ dramatically and I am really unsure how to deal with this observation and what to report to customers. Why do you prefer the spc-based scoring? I also noticed that the Gingras lab is still using spc stats heavily for their proximity labelling projects. Is there a consensus in the field why this still makes sense, especially given that for protein abundance estimation/differential expression we all moved away from this proxy (spc) years ago. Why is it still useful for AP-MS/PL?

tobiasko · 2021-12-16T10:34:16Z

tobiasko
Dec 16, 2021
Author

Dear @anesvi and @fcyu,

I fear we have to go two steps backwards! We have mayor problems running the base REPRINT workflow on Linux using the philosopher CLI (so let's focus on the spc data first!). I have attached the philosopher.yaml and the std.out/std.error in two zip archives.

Once the philosopher pipeline is done, we get the following picture:

tobiasko@fgcz-c-073:/scratch/FRAGPIPE/WU272904$ ls -l
total 333456
-rw-rw-rw-+ 1 bfabric SG_Employees  26973252 Dec 15 17:56 2021-12-15-decoys-reviewed-contam-UP000005640.fas
-rw-rw-rw-+ 1 bfabric SG_Employees  74942921 Dec 16 10:03 2021-12-15-decoys-reviewed-contam-UP000005640.fas.1.pepindex
-rw-rw-rw-+ 1 bfabric SG_Employees  38279511 Dec 16 10:03 2021-12-15-decoys-reviewed-contam-UP000005640.fas.2_-1207187027.peptide_idx_dict
-rw-rw-rw-+ 1 bfabric SG_Employees  74942988 Dec 16 10:03 2021-12-15-decoys-reviewed-contam-UP000005640.fas.2.pepindex
-rw-rw-rw-+ 1 bfabric SG_Employees   1319427 Dec 16 10:27 combined_peptide.tsv
-rw-rw-rw-+ 1 bfabric SG_Employees 117323437 Dec 16 10:26 combined.pep.xml
-rw-rw-rw-+ 1 bfabric SG_Employees    247684 Dec 16 10:28 combined_protein.tsv
-rw-rw-rw-+ 1 bfabric SG_Employees   7127018 Dec 16 10:26 combined.prot.xml
drwxr-xr-x+ 1 bfabric SG_Employees       774 Dec 16 10:27 CONTROL_1
drwxr-xr-x+ 1 bfabric SG_Employees       774 Dec 16 10:27 CONTROL_4
drwxr-xr-x+ 1 bfabric SG_Employees       774 Dec 16 10:27 CONTROL_5
drwxr-xr-x+ 1 bfabric SG_Employees       854 Dec 16 10:27 dCas9_PIPOX_2
drwxr-xr-x+ 1 bfabric SG_Employees       814 Dec 16 10:27 dCas9_PIPOX_3
drwxr-xr-x+ 1 bfabric SG_Employees       814 Dec 16 10:27 dCas9_PIPOX_6
-rw-rw-rw-+ 1 bfabric SG_Employees       878 Dec 16 10:08 Makefile
drwxr-xr-x+ 1 bfabric SG_Employees       734 Dec 16 10:27 P3F_7
drwxr-xr-x+ 1 bfabric SG_Employees       734 Dec 16 10:27 P3F_8
drwxr-xr-x+ 1 bfabric SG_Employees       734 Dec 16 10:27 P3F_9
-rw-rw-rw-+ 1 bfabric SG_Employees      7775 Dec 16 10:03 philosopher.1.txt
-rw-rw-rw-+ 1 bfabric SG_Employees      2834 Dec 16 10:03 philosopher.2.txt
-rw-rw-rw-+ 1 bfabric SG_Employees      6116 Dec 15 17:56 philosopher.yaml
-rw-rw-rw-+ 1 bfabric SG_Employees     90756 Dec 16 10:28 reprint.int.tsv
-rw-rw-rw-+ 1 bfabric SG_Employees     34860 Dec 16 10:28 reprint.spc.tsv
-rw-r--r--+ 1 bfabric SG_Employees    118810 Dec 16 10:43 WU272904.tmux-buffer.txt
-rw-r--r--+ 1 bfabric SG_Employees      6909 Dec 15 17:56 WU272904.yaml

But the reprint.spc.tsv is only initialised (it contains only zero entries):

tobiasko@fgcz-c-073:/scratch/FRAGPIPE/WU272904$ head reprint.spc.tsv 
PROTID	GENEID	PROTLEN	CONTROL_1_SPC	CONTROL_4_SPC	CONTROL_5_SPC	P3F_7_SPC	P3F_8_SPC	P3F_9_SPC	dCas9_PIPOX_2_SPC	dCas9_PIPOX_3_SPC	dCas9_PIPOX_6_SPC	
na	na	na	CONTROL	CONTROL	CONTROL	P3F	P3F	P3F	dCas9_PIPOX	dCas9_PIPOX	dCas9_PIPOX	
A1X283	SH3PXD2B	911	0	0	0	0	0	0	0	0	0	
A5YKK6	CNOT1	2376	0	0	0	0	0	0	0	0	0	
A6NHR9	SMCHD1	2005	0	0	0	0	0	0	0	0	0	
O00148	DDX39A	427	0	0	0	0	0	0	0	0	0	
O00264	PGRMC1	195	0	0	0	0	0	0	0	0	0	
O00267	SUPT5H	1087	0	0	0	0	0	0	0	0	0	
O00567	NOP56	594	0	0	0	0	0	0	0	0	0	
O14497	ARID1A	2285	0	0	0	0	0	0	0	0	0

In addition, we see some parsing errors for our FASTA-formated spike-in sequences (header fields are not separated!):

Q9Y4L1  HYOU1   999     0       0       0       0       0       0       0       0       0
Q7Z739  YTHDF3  585     0       0       0       0       0       0       0       0       0
Q9Y5A9  YTHDF2  579     0       0       0       0       0       0       0       0       0
Q9Y618  NCOR2   2514    0       0       0       0       0       0       0       0       0
up|P3F|P3F NoneDescription1             836     0       0       0       0       0       0       0       0       0
up|dCas9|dCas9 NoneDescription2         1403    0       0       0       0       0       0       0       0       0
O60315  ZEB2    1214    0       0       0       0       0       0       0       0       0
O75396  SEC22B  215     0       0       0       0       0       0       0       0       0
P02675  FGB     491     0       0       0       0       0       0       0       0       0
P08246  ELANE   267     0       0       0       0       0       0       0       0       0

The spike-in entries look like:

tobiasko@fgcz-c-073:/scratch/FRAGPIPE/WU272904$ tail 2021-12-15-decoys-reviewed-contam-UP000005640.fas
...
>up|dCas9|dCas9 NoneDescription2
MDYKDHDGDYKDHDIDYKDDDDKMAPKKKRKVGRGMDKKYSIGLAIGTNSVGWAVITDEYKVPSKKFKVLGNTDRHSIKKNLIGALLFDSGETAEATRLKRTARRRYTRRKNRICYLQEIFSNEMAKVDDSFFHRLEESFLVEEDKKHERHPIFGNIVDEVAYHEKYPTIYHLRKKLVDSTDKADLRLIYLALAHMIKFRGHFLIEGDLNPDNSDVDKLFIQLVQTYNQLFEENPINASGVDAKAILSARLSKSRRLENLIAQLPGEKKNGLFGNLIALSLGLTPNFKSNFDLAEDAKLQLSKDTYDDDLDNLLAQIGDQYADLFLAAKNLSDAILLSDILRVNTEITKAPLSASMIKRYDEHHQDLTLLKALVRQQLPEKYKEIFFDQSKNGYAGYIDGGASQEEFYKFIKPILEKMDGTEELLVKLNREDLLRKQRTFDNGSIPHQIHLGELHAILRRQEDFYPFLKDNREKIEKILTFRIPYYVGPLARGNSRFAWMTRKSEETITPWNFEEVVDKGASAQSFIERMTNFDKNLPNEKVLPKHSLLYEYFTVYNELTKVKYVTEGMRKPAFLSGEQKKAIVDLLFKTNRKVTVKQLKEDYFKKIECFDSVEISGVEDRFNASLGTYHDLLKIIKDKDFLDNEENEDILEDIVLTLTLFEDREMIEERLKTYAHLFDDKVMKQLKRRRYTGWGRLSRKLINGIRDKQSGKTILDFLKSDGFANRNFMQLIHDDSLTFKEDIQKAQVSGQGDSLHEHIANLAGSPAIKKGILQTVKVVDELVKVMGRHKPENIVIEMARENQTTQKGQKNSRERMKRIEEGIKELGSQILKEHPVENTQLQNEKLYLYYLQNGRDMYVDQELDINRLSDYDVDAIVPQSFLKDDSIDNKVLTRSDKNRGKSDNVPSEEVVKKMKNYWRQLLNAKLITQRKFDNLTKAERGGLSELDKAGFIKRQLVETRQITKHVAQILDSRMNTKYDENDKLIREVKVITLKSKLVSDFRKDFQFYKVREINNYHHAHDAYLNAVVGTALIKKYPKLESEFVYGDYKVYDVRKMIAKSEQEIGKATAKYFFYSNIMNFFKTEITLANGEIRKRPLIETNGETGEIVWDKGRDFATVRKVLSMPQVNIVKKTEVQTGGFSKESILPKRNSDKLIARKKDWDPKKYGGFDSPTVAYSVLVVAKVEKGKSKKLKSVKELLGITIMERSSFEKNPIDFLEAKGYKEVKKDLIIKLPKYSLFELENGRKRMLASAGELQKGNELALPSKYVNFLYLASHYEKLKGSPEDNEQKQLFVEQHKHYLDEIIEQISEFSKRVILADANLDKVLSAYNKHRDKPIREQAENIIHLFTLTNLGAPAAFKYFDTTIDRKRYTSTKEVLDATLIHQSITGLYETRIDLSQLGGD
>up|P3F|P3F NoneDescription1
MTTLAGAVPRMMRPGPGQNYPRSGFPLEVSTPLGQGRVNQLGGVFINGRPLPNHIRHKIVEMAHHGIRPCVISRQLRVSHGCVSKILCRYQETGSIRPGAIGGSKPKQVTTPDVEKKIEEYKRENPGMFSWEIRDKLLKDAVCDRNTVPSVSSISRILRSKFGKGEEEEADLERKEAEESEKKAKHSIDGILSERASAPQSDEGSDIDSEPDLPLKRKQRRSRTTFTAEQLEELERAFERTHYPDIYTREELAQRAKLTEARVQVWFSNRRARWRKQAGANQLMAFNHLIPGGFPPTAMPTLPTYQLSETSYQPTSIPQAVSDPSSTVHRPQPLPPSTVHQSTIPSNPDSSSAYCLPSTRHGFSSYTDSFVPPSGPSNPMNPTIGNGLSPQNSIRHNLSLHSKFIRVQNEGTGKSSWWMLNPEGGKSGKSPRRRAASMDNNSKFAKSRSRAAKKKASLQSGQEGAGDSPGSQFSKWPASPGSHSNDDFDNWSTFRPRTSSNASTISGRLSPIMTEQDDLGEGDVHSMVYPPSAAKMASTLPSLSEISNPENMENLLDNLNLLSSPTSLTVSTQSSPGTMMQQTPCYSFAPPNTSLNSPSPNYQKYTYGQSSMSPLPQMPIQTLQDNKSSYGGMSQYNCAPGLLKELLTSDSPPHNDIMTPVDPGVAQPNSRVLGQNVMMGPNSVMSTYGSQASHNKMMNPSSHTHPGHAQQTSAVNGRPLPHTVSTMPHTSGMNRLTQVKTPVQVPLPHPMQMSALGGYSSVSSCNGYGRMGLLHQEKLPSDLDGMFIERLDCDMESIIRNDLMDGDTLDFNFDNVLPNQSFPHSVKTTTHSWVSG

and were inserted by philosopher database ... --add

Can you give us a hint why the REPRINT pipeline fails?

philosopher.yaml.zip
WU272904.tmux-buffer.txt.zip

1 reply

tobiasko Dec 16, 2021
Author

Here is a listing of the pipeline parameters:

tobiasko@fgcz-c-073:/scratch/FRAGPIPE/WU272904$ cat philosopher.yaml 
analytics: true
slackToken: 
slackChannel: 
slackUserID: 
Steps:
  Database Search: yes
  Peptide Validation: yes
  PTM Localization: no
  Protein Inference: yes
  Label-Free Quantification: no
  Isobaric Quantification: no
  Bio Cluster Quantification: no
  FDR Filtering: yes
  Individual Reports: yes
  Integrated Reports: yes
  Integrated Isobaric Quantification: no
Database Search:
  protein_database: /scratch/FRAGPIPE/WU272904/2021-12-15-decoys-reviewed-contam-UP000005640.fas
  decoy_tag: rev_
  contaminant_tag: false
  search_engine: msfragger
  msfragger:
    path: /usr/local/nesvilab/MSFragger-3.4/MSFragger-3.4.jar
    memory: 128
    extension: raw
    num_threads: 64
    precursor_mass_lower: -20
    precursor_mass_upper: 20
    precursor_mass_units: 1
    data_type: 0
    precursor_true_tolerance: 20
    precursor_true_units: 1
    fragment_mass_tolerance: 20
    fragment_mass_units: 1
    calibrate_mass: 2
    use_all_mods_in_first_search: 0
    write_calibrated_mgf: 0
    isotope_error: 0/1/2
    mass_offsets: 0
    restrict_deltamass_to: all
    precursor_mass_mode: selected
    localize_delta_mass: 0
    delta_mass_exclude_ranges: (-1.5,3.5)
    fragment_ion_series: b,y
    ion_series_definitions: 
    search_enzyme_name_1: Trypsin
    search_enzyme_cut_1: KR
    search_enzyme_nocut_1: P
    allowed_missed_cleavage_1: 1
    search_enzyme_sense_1: C
    search_enzyme_name_2: 
    search_enzyme_cut_2: 
    search_enzyme_nocut_2: 
    allowed_missed_cleavage_2: 
    search_enzyme_sense_2: C
    num_enzyme_termini: 2
    clip_nTerm_M: 1
    variable_mod_01: 15.99490 M 3
    variable_mod_02: 42.01060 [^ 1
    variable_mod_03: 
    variable_mod_04: 
    variable_mod_05: 
    variable_mod_06: 
    variable_mod_07: 
    allow_multiple_variable_mods_on_residue: 0
    max_variable_mods_per_peptide: 3
    max_variable_mods_combinations: 5000
    mass_diff_to_variable_mod: 0
    output_file_extension: pepXML
    output_format: tsv_pepxml_pin
    output_report_topN: 1
    output_max_expect: 50
    report_alternative_proteins: 0
    precursor_charge: 1 4
    override_charge: 0
    digest_min_length: 7
    digest_max_length: 50
    digest_mass_range: 500.0 5000.0
    max_fragment_charge: 2
    track_zero_topN: 0
    zero_bin_accept_expect: 0
    zero_bin_mult_expect: 1
    add_topN_complementary: 0
    check_spectral_files: 1
    minimum_peaks: 15
    use_topN_peaks: 150
    deisotope: 1
    deneutralloss: 1
    min_fragments_modelling: 2
    min_matched_fragments: 4
    minimum_ratio: 0.01
    clear_mz_range: 0.0 0.0
    remove_precursor_peak: 0
    remove_precursor_range: -1.5,1.5
    intensity_transform: 0
    labile_search_mode: off
    diagnostic_intensity_filter: 0
    Y_type_masses: 
    diagnostic_fragments: 
    add_Cterm_peptide: 0.000000
    add_Cterm_protein: 0.000000
    add_Nterm_peptide: 0.000000
    add_Nterm_protein: 0.000000
    add_A_alanine: 0.000000
    add_C_cysteine: 57.021464
    add_D_aspartic_acid: 0.000000
    add_E_glutamic_acid: 0.000000
    add_F_phenylalanine: 0.000000
    add_G_glycine: 0.000000
    add_H_histidine: 0.000000
    add_I_isoleucine: 0.000000
    add_K_lysine: 0.000000
    add_L_leucine: 0.000000
    add_M_methionine: 0.000000
    add_N_asparagine: 0.000000
    add_P_proline: 0.000000
    add_Q_glutamine: 0.000000
    add_R_arginine: 0.000000
    add_S_serine: 0.000000
    add_T_threonine: 0.000000
    add_V_valine: 0.000000
    add_W_tryptophan: 0.000000
    add_Y_tyrosine: 0.000000
Peptide Validation:
  concurrent: false
  extension: pepXML
  clevel: 0
  accmass: true
  decoyprobs: true
  enzyme: trypsin
  exclude: false
  expectscore: true
  forcedistr: false
  glyc: false
  icat: false
  instrwarn: false
  leave: false
  maldi: false
  masswidth: 5
  minpeplen: 7
  minpintt: 2
  minpiprob: 0.9
  minprob: 0.05
  minrtntt: 2
  minrtprob: 0.9
  neggamma: false
  noicat: false
  nomass: false
  nonmc: false
  nonparam: true
  nontt: false
  optimizefval: false
  phospho: false
  pi: false
  ppm: true
  zero: false
PTM Localization:
  autodirect: false
  cions: 
  direct: false
  em: 2
  static: false
  fragppmtol: 15
  ifrags: false
  keepold: false
  lability: false
  massdiffmode: false
  excludemassdiffmin: 0
  excludemassdiffmax: 0
  massoffset: 0
  maxfragz: 0
  maxthreads: 4
  mino: 0
  minprob: 0
  mods: 
  nions: 
  nominofactor: false
  ppmtol: 1
  verbose: false
Protein Inference:
  accuracy: false
  allpeps: false
  confem: false
  delude: false
  excludezeros: false
  fpkm: false
  glyc: false
  icat: false
  instances: false
  iprophet: false
  logprobs: false
  maxppmdiff: 20
  minprob: 0.05
  mufactor: 1
  nogroupwts: false
  nonsp: false
  nooccam: false
  noprotlen: false
  normprotlen: false
  protmw: false
  softoccam: false
  unmapped: false
Label-Free Quantification:
  peakTimeWindow: 0.4
  retentionTimeWindow: 3
  tolerance: 10
  raw: false
  faims: false
Isobaric Quantification:
  bestPSM: false
  level: 2
  minProb: 0.7
  plex: 
  purity: 0.5
  removeLow: 0.0
  tolerance: 20
  uniqueOnly: false
  brand: tmt
  raw: false
Bio Cluster Quantification:
  organismUniProtID: 
  level: 0.9
FDR Filtering:
  psmFDR: 0.01
  peptideFDR: 0.01
  ionFDR: 0.01
  proteinFDR: 0.01
  peptideProbability: 0.7
  proteinProbability: 0.5
  peptideWeight: 1
  razor: false
  picked: false
  mapMods: false
  models: false
  sequential: false
Individual Reports:
  msstats: true
  withDecoys: false
  mzID: false
Integrated Reports:
  protein: true
  peptide: true
  proteinProbability: 0.9
  peptideProbability: 0.5
  uniqueOnly: false
  reprint: true
Integrated Isobaric Quantification:
  path: 
  memory: 6
  output: 
  channel_num: 10
  ref_tag: Bridge
  groupby: -1
  psm_norm: false
  outlier_removal: true
  prot_norm: -1
  min_pep_prob: 0.9
  min_purity: 0.5
  min_percent: 0.05
  unique_pep: false
  unique_gene: 0
  best_psm: true
  prot_exclude: none
  allow_overlabel: true
  allow_unlabeled: true
  mod_tag: none
  min_site_prob: -1
  ms1_int: true
  top3_pep: true
  print_RefInt: false
  add_Ref: -1
  max_pep_prob_thres: 0
  min_ntt: 0
  aggregation_method: 0

anesvi · 2021-12-16T13:40:16Z

anesvi
Dec 16, 2021
Maintainer

This seem to be philosopher issue. So Felipe needs to investigate From: Tobias Kockmann ***@***.***> Sent: Thursday, December 16, 2021 5:34 AM To: Nesvilab/philosopher ***@***.***> Cc: Nesvizhskii, Alexey ***@***.***>; Mention ***@***.***> Subject: Re: [Nesvilab/philosopher] REPRINT analysis (Discussion #298) External Email - Use Caution Dear @anesvi<https://github.com/anesvi> and @fcyu<https://github.com/fcyu>, I fear we have to go two steps backwards! We have mayor problems running the base REPRINT workflow on Linux using the philosopher CLI (so let's focus on the spc data first!). I have attached the philosopher.yaml and the std.out/std.error in two zip archives. Once the philosopher pipeline is done, we get the following picture: ***@***.***:/scratch/FRAGPIPE/WU272904$ ls -l total 333456

-rw-rw-rw-+ 1 bfabric SG_Employees 26973252 Dec 15 17:56 2021-12-15-decoys-reviewed-contam-UP000005640.fas

-rw-rw-rw-+ 1 bfabric SG_Employees 74942921 Dec 16 10:03 2021-12-15-decoys-reviewed-contam-UP000005640.fas.1.pepindex

-rw-rw-rw-+ 1 bfabric SG_Employees 38279511 Dec 16 10:03 2021-12-15-decoys-reviewed-contam-UP000005640.fas.2_-1207187027.peptide_idx_dict

-rw-rw-rw-+ 1 bfabric SG_Employees 74942988 Dec 16 10:03 2021-12-15-decoys-reviewed-contam-UP000005640.fas.2.pepindex

-rw-rw-rw-+ 1 bfabric SG_Employees 1319427 Dec 16 10:27 combined_peptide.tsv

-rw-rw-rw-+ 1 bfabric SG_Employees 117323437 Dec 16 10:26 combined.pep.xml

-rw-rw-rw-+ 1 bfabric SG_Employees 247684 Dec 16 10:28 combined_protein.tsv

-rw-rw-rw-+ 1 bfabric SG_Employees 7127018 Dec 16 10:26 combined.prot.xml drwxr-xr-x+ 1 bfabric SG_Employees 774 Dec 16 10:27 CONTROL_1 drwxr-xr-x+ 1 bfabric SG_Employees 774 Dec 16 10:27 CONTROL_4 drwxr-xr-x+ 1 bfabric SG_Employees 774 Dec 16 10:27 CONTROL_5 drwxr-xr-x+ 1 bfabric SG_Employees 854 Dec 16 10:27 dCas9_PIPOX_2 drwxr-xr-x+ 1 bfabric SG_Employees 814 Dec 16 10:27 dCas9_PIPOX_3 drwxr-xr-x+ 1 bfabric SG_Employees 814 Dec 16 10:27 dCas9_PIPOX_6

-rw-rw-rw-+ 1 bfabric SG_Employees 878 Dec 16 10:08 Makefile drwxr-xr-x+ 1 bfabric SG_Employees 734 Dec 16 10:27 P3F_7 drwxr-xr-x+ 1 bfabric SG_Employees 734 Dec 16 10:27 P3F_8 drwxr-xr-x+ 1 bfabric SG_Employees 734 Dec 16 10:27 P3F_9

-rw-rw-rw-+ 1 bfabric SG_Employees 7775 Dec 16 10:03 philosopher.1.txt

-rw-rw-rw-+ 1 bfabric SG_Employees 2834 Dec 16 10:03 philosopher.2.txt

-rw-rw-rw-+ 1 bfabric SG_Employees 6116 Dec 15 17:56 philosopher.yaml

-rw-rw-rw-+ 1 bfabric SG_Employees 90756 Dec 16 10:28 reprint.int.tsv

-rw-rw-rw-+ 1 bfabric SG_Employees 34860 Dec 16 10:28 reprint.spc.tsv

-rw-r--r--+ 1 bfabric SG_Employees 118810 Dec 16 10:43 WU272904.tmux-buffer.txt

-rw-r--r--+ 1 bfabric SG_Employees 6909 Dec 15 17:56 WU272904.yaml But the reprint.spc.tsv is only initialised (it contains only zero entries): ***@***.***:/scratch/FRAGPIPE/WU272904$ head reprint.spc.tsv PROTID GENEID PROTLEN CONTROL_1_SPC CONTROL_4_SPC CONTROL_5_SPC P3F_7_SPC P3F_8_SPC P3F_9_SPC dCas9_PIPOX_2_SPC dCas9_PIPOX_3_SPC dCas9_PIPOX_6_SPC na na na CONTROL CONTROL CONTROL P3F P3F P3F dCas9_PIPOX dCas9_PIPOX dCas9_PIPOX A1X283 SH3PXD2B 911 0 0 0 0 0 0 0 0 0 A5YKK6 CNOT1 2376 0 0 0 0 0 0 0 0 0 A6NHR9 SMCHD1 2005 0 0 0 0 0 0 0 0 0 O00148 DDX39A 427 0 0 0 0 0 0 0 0 0 O00264 PGRMC1 195 0 0 0 0 0 0 0 0 0 O00267 SUPT5H 1087 0 0 0 0 0 0 0 0 0 O00567 NOP56 594 0 0 0 0 0 0 0 0 0 O14497 ARID1A 2285 0 0 0 0 0 0 0 0 0 In addition, we see some parsing errors for our FASTA-formated spike-in sequences (header fields are not separated!): Q9Y4L1 HYOU1 999 0 0 0 0 0 0 0 0 0 Q7Z739 YTHDF3 585 0 0 0 0 0 0 0 0 0 Q9Y5A9 YTHDF2 579 0 0 0 0 0 0 0 0 0 Q9Y618 NCOR2 2514 0 0 0 0 0 0 0 0 0 up|P3F|P3F NoneDescription1 836 0 0 0 0 0 0 0 0 0 up|dCas9|dCas9 NoneDescription2 1403 0 0 0 0 0 0 0 0 0 O60315 ZEB2 1214 0 0 0 0 0 0 0 0 0 O75396 SEC22B 215 0 0 0 0 0 0 0 0 0 P02675 FGB 491 0 0 0 0 0 0 0 0 0 P08246 ELANE 267 0 0 0 0 0 0 0 0 0 The spike-in entries look like: ***@***.***:/scratch/FRAGPIPE/WU272904$ tail 2021-12-15-decoys-reviewed-contam-UP000005640.fas ...

up|dCas9|dCas9 NoneDescription2

MDYKDHDGDYKDHDIDYKDDDDKMAPKKKRKVGRGMDKKYSIGLAIGTNSVGWAVITDEYKVPSKKFKVLGNTDRHSIKKNLIGALLFDSGETAEATRLKRTARRRYTRRKNRICYLQEIFSNEMAKVDDSFFHRLEESFLVEEDKKHERHPIFGNIVDEVAYHEKYPTIYHLRKKLVDSTDKADLRLIYLALAHMIKFRGHFLIEGDLNPDNSDVDKLFIQLVQTYNQLFEENPINASGVDAKAILSARLSKSRRLENLIAQLPGEKKNGLFGNLIALSLGLTPNFKSNFDLAEDAKLQLSKDTYDDDLDNLLAQIGDQYADLFLAAKNLSDAILLSDILRVNTEITKAPLSASMIKRYDEHHQDLTLLKALVRQQLPEKYKEIFFDQSKNGYAGYIDGGASQEEFYKFIKPILEKMDGTEELLVKLNREDLLRKQRTFDNGSIPHQIHLGELHAILRRQEDFYPFLKDNREKIEKILTFRIPYYVGPLARGNSRFAWMTRKSEETITPWNFEEVVDKGASAQSFIERMTNFDKNLPNEKVLPKHSLLYEYFTVYNELTKVKYVTEGMRKPAFLSGEQKKAIVDLLFKTNRKVTVKQLKEDYFKKIECFDSVEISGVEDRFNASLGTYHDLLKIIKDKDFLDNEENEDILEDIVLTLTLFEDREMIEERLKTYAHLFDDKVMKQLKRRRYTGWGRLSRKLINGIRDKQSGKTILDFLKSDGFANRNFMQLIHDDSLTFKEDIQKAQVSGQGDSLHEHIANLAGSPAIKKGILQTVKVVDELVKVMGRHKPENIVIEMARENQTTQKGQKNSRERMKRIEEGIKELGSQILKEHPVENTQLQNEKLYLYYLQNGRDMYVDQELDINRLSDYDVDAIVPQSFLKDDSIDNKVLTRSDKNRGKSDNVPSEEVVKKMKNYWRQLLNAKLITQRKFDNLTKAERGGLSELDKAGFIKRQLVETRQITKHVAQILDSRMNTKYDENDKLIREVKVITLKSKLVSDFRKDFQFYKVREINNYHHAHDAYLNAVVGTALIKKYPKLESEFVYGDYKVYDVRKMIAKSEQEIGKATAKYFFYSNIMNFFKTEITLANGEIRKRPLIETNGETGEIVWDKGRDFATVRKVLSMPQVNIVKKTEVQTGGFSKESILPKRNSDKLIARKKDWDPKKYGGFDSPTVAYSVLVVAKVEKGKSKKLKSVKELLGITIMERSSFEKNPIDFLEAKGYKEVKKDLIIKLPKYSLFELENGRKRMLASAGELQKGNELALPSKYVNFLYLASHYEKLKGSPEDNEQKQLFVEQHKHYLDEIIEQISEFSKRVILADANLDKVLSAYNKHRDKPIREQAENIIHLFTLTNLGAPAAFKYFDTTIDRKRYTSTKEVLDATLIHQSITGLYETRIDLSQLGGD

up|P3F|P3F NoneDescription1

MTTLAGAVPRMMRPGPGQNYPRSGFPLEVSTPLGQGRVNQLGGVFINGRPLPNHIRHKIVEMAHHGIRPCVISRQLRVSHGCVSKILCRYQETGSIRPGAIGGSKPKQVTTPDVEKKIEEYKRENPGMFSWEIRDKLLKDAVCDRNTVPSVSSISRILRSKFGKGEEEEADLERKEAEESEKKAKHSIDGILSERASAPQSDEGSDIDSEPDLPLKRKQRRSRTTFTAEQLEELERAFERTHYPDIYTREELAQRAKLTEARVQVWFSNRRARWRKQAGANQLMAFNHLIPGGFPPTAMPTLPTYQLSETSYQPTSIPQAVSDPSSTVHRPQPLPPSTVHQSTIPSNPDSSSAYCLPSTRHGFSSYTDSFVPPSGPSNPMNPTIGNGLSPQNSIRHNLSLHSKFIRVQNEGTGKSSWWMLNPEGGKSGKSPRRRAASMDNNSKFAKSRSRAAKKKASLQSGQEGAGDSPGSQFSKWPASPGSHSNDDFDNWSTFRPRTSSNASTISGRLSPIMTEQDDLGEGDVHSMVYPPSAAKMASTLPSLSEISNPENMENLLDNLNLLSSPTSLTVSTQSSPGTMMQQTPCYSFAPPNTSLNSPSPNYQKYTYGQSSMSPLPQMPIQTLQDNKSSYGGMSQYNCAPGLLKELLTSDSPPHNDIMTPVDPGVAQPNSRVLGQNVMMGPNSVMSTYGSQASHNKMMNPSSHTHPGHAQQTSAVNGRPLPHTVSTMPHTSGMNRLTQVKTPVQVPLPHPMQMSALGGYSSVSSCNGYGRMGLLHQEKLPSDLDGMFIERLDCDMESIIRNDLMDGDTLDFNFDNVLPNQSFPHSVKTTTHSWVSG and were inserted by philosopher database ... --add Can you give as a hint why the REPRINT pipeline fails? philosopher.yaml.zip<https://github.com/Nesvilab/philosopher/files/7726424/philosopher.yaml.zip> WU272904.tmux-buffer.txt.zip<https://github.com/Nesvilab/philosopher/files/7726425/WU272904.tmux-buffer.txt.zip> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#298 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIIMM6YICJ65I6H77D4PZ73URG6DHANCNFSM5JZE5SKQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

7 replies

tobiasko Dec 16, 2021
Author

It unfortunately did not change the results in the reprint.spc.tsv (btw: the example file listed here says Protein Inference: yes). Which email should I use to send you a link to the archived folder?

tobiasko Dec 16, 2021
Author

Regarding the parsing of FASTA headers: We were assuming that philosopher strictly sticks to the uniprotKB requirements:

>db|UniqueIdentifier|EntryName ProteinName OS=OrganismName OX=OrganismIdentifier [GN=GeneName ]PE=ProteinExistence SV=SequenceVersion

That means before the first white space there is 3 fields (db, UniqueIdentifier, EntryName) separated by |. The first field after the first white space is the ProteinName. What your internal parsing rule/mechanism?

prvst Dec 16, 2021
Collaborator

The program uses some header elements and some regex to guess what's the header, for the Uniprot it tries to find things like sp or tr at the beginning, if it cannot match to any regex model, it defaults to "generic"

tobiasko Dec 17, 2021
Author

ok. That means our default approach to watermark spike-in sequences by using the db field of the fasta header will not work with philosopher. DAAAM. In general, is there any option to control how philosopher parses fasta files?

prvst Dec 17, 2021
Collaborator

There's no problem in having the db in the header, but the text won't be parsed as an Uniprot entry because there is no db in their headers. The db from the example above is to mark the position for either sp or tr. We have some guidelines here, including a link to the Uniprot website explaining their format.

cpanse · 2021-12-16T14:42:36Z

cpanse
Dec 16, 2021

@prvst btw, our current 'FragPipe' versions are:
IonQuant (1.7.17)
MSFragger (3.4)
philosopher (4.1.1)

0 replies

tobiasko · 2021-12-16T15:48:06Z

tobiasko
Dec 16, 2021
Author

Out of curiosity I also tested the same raw files and FASTA DB using FragPipe GUI v17.1 using the default workflow template and the reprint.spc.tsv looks normal.

10 replies

tobiasko Dec 23, 2021
Author

Hi Fengchao,

that is really kind! Is that included in the 17.2-build4? Just downloaded it to test the headless mode (CLI).

Best,
Tobi

fcyu Dec 23, 2021
Maintainer

Hi Tobi,

No, it is not. The new IonQuant is not included in 17.2-build4. I will send you a new one later.

Best,

Fengchao

cpanse Dec 23, 2021

Hi Tobi,

No, it is not. The new IonQuant is not included in 17.2-build4. I will send you a new one later.

Best,

Fengchao

Dear @fcyu; can you please put me CC. Thanks and Merry Xmas. Christian

fcyu Dec 23, 2021
Maintainer

Hi @cpanse ,

Sure,

Best,

Fengchao

fcyu Dec 23, 2021
Maintainer

Hi @tobiasko and @cpanse ,

I am in the middle of adding a new module for the label quant in IonQuant. I wanted to send you the new FragPipe after that. But it looks like I should send you the version with reprint file fix soon. So, here (https://www.dropbox.com/s/kewu5bscaqq6awv/FragPipe-17.2-build7.zip?dl=1) it is. It does not contain the label quant related part I am working on, but I think it should not affect your project.

In case you wonder, it has the headless part (all versions later than 17.2-build4 have headless support).

Please feel free to contact us if you have any questions.

Best,

Fengchao

anesvi · 2021-12-17T14:18:12Z

anesvi
Dec 17, 2021
Maintainer

For now can just enable it in the Validation tab (there is a checkbox) From: Fengchao ***@***.***> Sent: Friday, December 17, 2021 8:58 AM To: Nesvilab/philosopher ***@***.***> Cc: Nesvizhskii, Alexey ***@***.***>; Mention ***@***.***> Subject: Re: [Nesvilab/philosopher] REPRINT analysis (Discussion #298) External Email - Use Caution I think I know why the reprint.spec.tsv is missing. We disabled “generate protein level report” in LFQ-MBR workflow because Philosopher needs lots of memory for that if there are many experiments. I am supposed to let IonQuant to generate it, but I forgot. Sorry about that. Will fix this in the next release. As to the format, I will investigate and get back to you soon. Best, Fengchao — Reply to this email directly, view it on GitHub<#298 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIIMM6ZW4CY3RQ4ZZWA7VV3URM6WXANCNFSM5JZE5SKQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

0 replies

anesvi · 2021-12-17T14:19:14Z

anesvi
Dec 17, 2021
Maintainer

Thanks, Need to fix that, so all files have GENEID and PoroteinID For the intensity file, LENGTH is not needed because SAINT does not use it with intensity data From: Tobias Kockmann ***@***.***> Sent: Friday, December 17, 2021 7:04 AM To: Nesvilab/philosopher ***@***.***> Cc: Nesvizhskii, Alexey ***@***.***>; Mention ***@***.***> Subject: Re: [Nesvilab/philosopher] REPRINT analysis (Discussion #298) External Email - Use Caution In addition: reprint.spc.tsv contains the columns, PROTID, GENEID, PROTLEN. reprint.int.tsv written by the LFQ-MBR lacks GENEID and PROTLEN. And reprint.int.tsv written by default contains PROTID and GENEID. So 3 different version of a reprint file layout. — Reply to this email directly, view it on GitHub<#298 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIIMM65QVHCHS4JQKOTZMKDURMRJPANCNFSM5JZE5SKQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> ********************************************************** Electronic Mail is not secure, may not be read every day, and should not be used for urgent or sensitive issues

1 reply

tobiasko Dec 17, 2021
Author

The reprint.int.tsv written by the FragPipe GUI (LFQ-MBR) only contains the PROTID column plus the sample-wise intensity columns:

$ head reprint.int.tsv
PROTID  CONTROL_5_INT   CONTROL_6_INT   CONTROL_9_INT   P3F_1_INT       P3F_2_IN                    T       P3F_3_INT       dCas9_PIPOX_4_INT       dCas9_PIPOX_7_INT       dCas9_PI                    POX_8_INT
na      CONTROL CONTROL CONTROL P3F     P3F     P3F     dCas9_PIPOX     dCas9_PI                    POX     dCas9_PIPOX
A1X283  3.6032464E7     3.6481436E7     3.4943484E7     0       0       0      6                    .1442028E7      3.2160226E7     5.4379384E7
A5YKK6  3.1144484E7     2.63871E7       5.2027896E7     0       0       0      4                    .858428E7       2.3509526E7     5.1186152E7
A6NHQ2  5.94209E7       6.5754752E7     8.2572232E7     6.3045384E7     6.504536                    8E7     7.0902664E7     4.8055856E7     6.641566E7      5.4153604E7
A6NHR9  1.58107856E8    1.35872064E8    1.54410368E8    7.6190952E7     4.174058                    4E7     0       1.2284208E8     1.24031648E8    1.43661056E8
A6NKT7  1.00284112E8    7.7752736E7     1.02533672E8    0       0       7.981154                    4E7     7.9495456E7     1.00284096E8    9.7970448E7
A8CG34  3060797.2       0       1380756.6       0       0       4865289.0      0                    0       1565045.2
A8MTJ3  3.7011616E7     7.3714608E7     3.9313956E7     3.9720056E7     0      0                    3.3546914E7     2.410341E7      2.814414E7
B2RPK0  1.35720592E8    1.94438528E8    1.72738144E8    1.03993896E8    4.544512                    8E7     4.4537528E7     1.17095536E8    1.16183152E8    9.963096E7

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

REPRINT analysis #298

{{title}}

Replies: 8 comments 21 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

REPRINT analysis #298

tobiasko Dec 10, 2021

Replies: 8 comments · 21 replies

anesvi Dec 10, 2021 Maintainer

fcyu Dec 10, 2021 Maintainer

anesvi Dec 10, 2021 Maintainer

tobiasko Dec 17, 2021 Author

tobiasko Dec 16, 2021 Author

tobiasko Dec 16, 2021 Author

anesvi Dec 16, 2021 Maintainer

tobiasko Dec 16, 2021 Author

tobiasko Dec 16, 2021 Author

prvst Dec 16, 2021 Collaborator

tobiasko Dec 17, 2021 Author

prvst Dec 17, 2021 Collaborator

cpanse Dec 16, 2021

tobiasko Dec 16, 2021 Author

tobiasko Dec 23, 2021 Author

fcyu Dec 23, 2021 Maintainer

cpanse Dec 23, 2021

fcyu Dec 23, 2021 Maintainer

fcyu Dec 23, 2021 Maintainer

anesvi Dec 17, 2021 Maintainer

anesvi Dec 17, 2021 Maintainer

tobiasko Dec 17, 2021 Author

tobiasko
Dec 10, 2021

Replies: 8 comments 21 replies

anesvi
Dec 10, 2021
Maintainer

fcyu Dec 10, 2021
Maintainer

anesvi
Dec 10, 2021
Maintainer

tobiasko Dec 17, 2021
Author

tobiasko
Dec 16, 2021
Author

tobiasko Dec 16, 2021
Author

anesvi
Dec 16, 2021
Maintainer

tobiasko Dec 16, 2021
Author

tobiasko Dec 16, 2021
Author

prvst Dec 16, 2021
Collaborator

tobiasko Dec 17, 2021
Author

prvst Dec 17, 2021
Collaborator

cpanse
Dec 16, 2021

tobiasko
Dec 16, 2021
Author

tobiasko Dec 23, 2021
Author

fcyu Dec 23, 2021
Maintainer

fcyu Dec 23, 2021
Maintainer

fcyu Dec 23, 2021
Maintainer

anesvi
Dec 17, 2021
Maintainer

anesvi
Dec 17, 2021
Maintainer

tobiasko Dec 17, 2021
Author