Skip to content

Latest commit

 

History

History
39 lines (24 loc) · 925 Bytes

README.md

File metadata and controls

39 lines (24 loc) · 925 Bytes
Raw

Translation Tasks

Paper

Citation

Groups and Tasks

Groups

  • gpt3_translation_tasks
  • wmt14
  • wmt16
  • wmt20
  • iwslt2017

Tasks

Checklist

For adding novel benchmarks/datasets to the library:

  • Is the task an existing benchmark in the literature?
    • Have you referenced the original paper that introduced the task?
    • If yes, does the original paper provide a reference implementation? If so, have you checked against the reference implementation and documented how to run such a test?

If other tasks on this dataset are already supported:

  • Is the "Main" variant of this task clearly denoted?
  • Have you provided a short sentence in a README on what each new variant adds / evaluates?
  • Have you noted which, if any, published evaluation setups are matched by this variant?
    • Checked for equivalence with v0.3.0 LM Evaluation Harness