The ATI Challenge was a one-time event hosted by QDR and has ended. The announcement below remains available for documentation purposes. Please contact qdr if you are interested in using Annotation for Transparent Inquiry.
Overview
QDR invites scholars in the social and health sciences, humanities, law, and others disciplines that employ qualitative data and methods, to submit proposals to the Annotation for Transparent Inquiry (ATI) Challenge. QDR seeks proposals from scholars who are currently writing a manuscript that they plan to submit to a leading journal in their respective discipline.
Scholars selected to participate will use cutting-edge annotation technology to enhance their manuscripts. They will play a key role in demonstrating how qualitative and multi-method researchers can join and benefit from academia’s ongoing transparency revolution.
Submissions to the ATI Challenge will be judged meritocratically. ATI Challenge participants will receive an award of $2,000 (as either an honorarium or research support), subject to any relevant tax and visa status limitations. They will also attend a workshop in New York City in November 2018, where they will meet and network with other scholars who share a commitment to making their research accessible and evaluable. All travel and accommodation expenses will be covered by QDR.
Timeline
- May 31, 2018 – Proposal submissions due
- July 13, 2018 – Notification of status of application
- October 26, 2018 – Manuscript and annotations due
- November 29-30, 2018 – Workshop in New York City
ATI
QDR and Hypothesis have partnered to develop a new approach to transparency in qualitative and multi-method research: Annotation for Transparent Inquiry (ATI). Scholars who use ATI enrich their work by linking supporting data sources, excerpts from those sources, and “analytic notes” discussing data generation and analysis directly to a digital manuscript. By accelerating access to research-enhancing information and easing readability, ATI augments and amplifies the impact of qualitative research.
ATI builds on “active citation” – an earlier approach to achieving transparency in qualitative research pioneered by Andrew Moravcsik. ATI employs “open annotation,” which allows for the generation, sharing, and discovery of digital annotations across the web.
More information is available on the QDR site – you can see a visual representation of ATI on the ATI At a Glance page, see links to a few early prototypes, and read more about ATI’s foundations and goals on the Why ATI? page.
The ATI Initiative
The ATI Challenge is part of the broader ATI Initiative that QDR launched in 2017, in partnership with Hypothes.is and Cambridge University Press, and with generous funding from the Robert Wood Johnson Foundation and the National Science Foundation. The ATI Initiative provides QDR and its partners an opportunity to assess the effectiveness of this exciting new approach to openness, and to receive feedback on the technology that powers it.
The ATI Initiative entails two workshops. The first workshop took place on February 22 and 23, 2018 in New York City. It focused on articles based on qualitative evidence that were recently published, or were forthcoming, in top-tier academic journals. Authors retrospectively annotated the articles using ATI. Articles and links to annotations can be found here. The second workshop – with which the ATI Challenge is associated – will evaluate manuscripts that scholars annotated (using ATI) while writing them.
The ATI Challenge
Faculty and advanced graduate students who are interested in participating in the ATI Challenge are encouraged to submit a proposal for consideration by the ATI Challenge Selection Committee. Proposals should be submitted as a Word document or PDF via email to qdr@syr.edu . The proposal deadline was May 31, 2018 (extended from May 11).
Proposals should include the following:
- A current CV
- An abstract of the manuscript you plan to write (250 words)
- A statement indicating at what stage your research and writing stand (250 words)
- A “Data Overview” describing the main types of qualitative evidence and methods the manuscript will employ and any challenges you foresee in sharing data as part of your annotations (250 words)
- A description of how you anticipate engaging in annotation – what types of passages you’ll annotate and the kinds of information you will include in the various elements of your annotations (250 words)
Participants will be selected by a committee drawn from multiple disciplines in the social and health sciences, humanities, education, and law, comprising faculty with deep experience conducting qualitative and multi-method research. Members of this committee will evaluate all proposals received by the May 31, 2018 due date, and will accept up to 15 proposals for inclusion in the ATI Challenge and for discussion at the November 2018 workshop. Applicants will be notified of the status of their proposal by July 13, 2018 (extended from June 18 due to strong interest).
Our goal is to choose a set of manuscripts (and associated annotations) that will help us to identify the different ways in which ATI contributes to academic research and publishing, and to identify the challenges that arise in using ATI. Accordingly, we will seek to identify:
- A set of manuscripts that collectively employ multiple analytic and interpretive approaches, use diverse types of qualitative data, and examine various world regions and temporal periods;
- Manuscripts that seem likely to be significantly enriched through the use of ATI;
- Manuscripts that present annotation difficulties with which QDR and partners have not yet grappled and thus for which a use case is helpful; and
- Proposals that are methodologically sound, clear, complete, and convincing.
Each author whose proposal is accepted for the ATI Challenge is expected to complete their manuscript, generate all of their annotations, and submit both the manuscript and the annotations to QDR by October 26, 2018. If they consent to do so, authors will also be asked to keep an “ATI logbook” and complete a brief questionnaire chronicling their experiences using ATI. QDR personnel will be available to assist and answer questions as authors annotate their work, including by phone/Skype.
QDR will curate and assemble each project, archive its data sources, and use Hypothesis technology to generate open annotations. With regard to pre-workshop access to the annotated manuscripts, authors have two choices. They may restrict access to members of the ATI Pilot Working Group (in which case the project will be posted to a password protected page on the QDR website). Alternatively, they may choose to have their project more widely available (in which case it will be posted to a pre-print server). Posting to either venue will occur by November 12, 2018.
Authors will be asked to nominate an advanced graduate student or faculty member from a graduate program other than their own to evaluate their annotation project. Reviewers will evaluate the use of ATI (not the manuscript itself), using a rubric we have created, by November 21, 2018. Reviews will be shared with authors on November 26, 2018 in advance of the November 29-30, 2018 workshop.
Authors who restricted access to their projects to ATI Pilot Working Group members in advance of the workshop may continue to keep their projects “private” as they seek to place the manuscript in a journal. If the manuscript has not been accepted for publication by 18 months after the workshop, however, we will post an annotated version of the manuscript to a pre-print-server.
Each author’s ATI Data Supplement (a compilation of the Data Overview and annotations with associated data sources), which will be citable with a digital object identifier (DOI), will be published on the QDR site when the article is published by a journal or the paper is published on a pre-print server.
The author(s) of each manuscript will receive an honorarium of $2,000. (Co-authors may share this honorarium in whatever way they see fit.) Reviewers will receive an honorarium of $1,000.
Potential participants are encouraged to contact QDR (qdr@syr.edu) to discuss their envisioned manuscripts and annotations and to ask any questions they may have.
FAQs
What is ATI?
ATI is a new approach to openness in qualitative and multi-method research that empowers scholars to enhance their published work through “open annotation,” linking particular passages in the text to an analytic note, excerpt, underlying data source, and full citation.
Is ATI the only technique that can be used to make more transparent research that employs qualitative data and research methods?
No, ATI is certainly not the only way to achieve research transparency in, and share data associated with, qualitative research, nor is it suitable for all types of qualitative scholarship. That said, we think ATI is a promising approach to openness in qualitative and multi-method research, and that the technology that underlies ATI is very exciting!
Annotations can have four elements – an analytic note, one or more excerpts, a link to one or more data sources, and one or more citations. In what order should I include these?
The first two elements of your annotation should be the analytic note and the source excerpt; you may include these in whatever order you see fit. We recommend that you think about the overall purpose of the annotation when you are considering the order. The data source should always be the penultimate element in the annotations, and the citation should go last.
All of my sources are confidential and/or under copyright. Can I still use ATI and submit a proposal for the ATI Challenge?
Yes. Please be sure to note in your proposal any difficulties you foresee with sharing the data that underlie your manuscript, and to describe how those difficulties may affect your use of ATI. Even if you cannot share all of the data arising from your research, providing excerpts from data sources that are under constraint, and using analytic notes, can enhance your manuscript. QDR is happy to answer questions you have in this regard; we also offer some guidance on the QDR site regarding sharing data generated through human participants research, and data that are under copyright.
My work is more theoretical than empirical. Can I still submit a proposal to the ATI Challenge?
Maybe. Some more theoretical work can certainly benefit from using ATI. We could imagine, for instance, ATI working very effectively to enhance a political theory manuscript that relies heavily on close textual analysis.
I am a quantitative researcher. Can I participate in the ATI Challenge?
Maybe. ATI is designed for qualitative research. If you do multi-method work and your article contains significant qualitative portions based on original research, we would be happy to receive a proposal from you.
I want to annotate an article that has already been published. Can I still participate in the ATI Challenge?
Unfortunately not. The “ATI Challenge” aspect of the ATI Initiative will only include scholars who annotate a manuscript while writing it. However, you can use ATI to annotate any web page. Thus you can annotate your published article using ATI at any time; QDR would be glad to assemble the elements of your annotations and create the digital overlay for your article.
If my proposal is selected for the ATI Challenge, can I submit the manuscript to a journal during the course of 2018, and even before the November 2018 workshop?
Yes. Your participation in the ATI Challenge and your scholarly publishing pursuits are independent. If you complete your manuscript in the course of the ATI Challenge, you should feel free to submit it to the journal of your choice. The annotations you would be creating as part of the ATI Challenge would serve as supplemental materials to your article.
If I want the annotations I create for my article to be considered as part of the journal review process, can QDR help me with a private version of the annotations?
Yes. We can create a private link to an anonymized version of the annotations for the editor to provide to the journal’s reviewers. We have no influence, however, on whether editors are willing to include annotations in the review process.
Once my article is accepted for publication, will QDR help me to include the annotations with my article as it is published on the journal web site?
Yes – we will make our best efforts to help. If the journal won’t let us display the annotations we’ll make public the pre-print version of the manuscript (or the version published on the QDR site), with the annotations, for demonstration and learning purposes.
If my proposal to participate in the ATI Challenge is rejected, can I still annotate my manuscript anyway?
Yes, absolutely! QDR would be glad to assemble the elements of your annotations and create the digital overlay for your manuscript (should you publish it on a pre-prints server) and for your article (on the journal website) once you have placed it.
Do I need the permission of editor of the journal to which I hope to send my manuscript in order to participate?
No, but you might need permission to have your ATI data supplement included as supplemental materials accompanying your article; you should check with your editor once your piece is accepted for publication.