Overview
The Qualitative Data Repository (QDR, www.qdr.org) invites proposals from scholars whose work focuses on some aspect of law, law and society, legal institutions, or other law-related topics to pilot a new tool for making qualitative and multi-method research more transparent. The “Anno-REP” tool allows scholars to restructure, edit, and package annotations that enhance their digital scholarship. In particular, the tool facilitates the use of “annotation for transparent inquiry”, a new approach to making qualitative and multi-method research more transparent. Scholars whose proposals are selected for inclusion in the pilot will use the Anno-REP tool to annotate a manuscript based on qualitative or multi-method research that they are currently writing and plan to submit to a leading journal.
Participants will receive an honorarium of $1,500 and will gather at a workshop in April 2022. QDR will cover travel and lodging expenses, with gratitude to the National Science Foundation Law and Science program (award number 1946272).
Timeline
- January 12, 2022 – Proposals due
- January 19, 2022 -- Decisions on proposals announced
- April 4, 2022 – Participants’ manuscripts with annotations and author logs due
- April 28-29, 2022 – Workshop in New York City (Note: We anticipate offering a hybrid model workshop with most attendees present in NYC, but with an option for remote participation (e.g. via Zoom))
ATI
Annotation for Transparent Inquiry (ATI) is a new approach to transparency developed by QDR and the software nonprofit Hypothesis (https://hypothes.is/). Scholars who use ATI enrich their work by linking supporting data sources, excerpts from those sources, and “analytic notes” discussing data generation and analysis directly to a digital manuscript. ATI builds on an earlier approach to achieving transparency in qualitative research pioneered by Moravcsik (“active citation”, e.g., 2016). ATI employs “open annotation,” which allows for the generation, sharing, and discovery of digital annotations across the web (Sanderson et al. 2017). You can read more about ATI on the “ATI at a Glance” page on the QDR site, which includes links to a few early examples. Basic instructions for using ATI can be found here. By accelerating access to research-enhancing information and easing readability, ATI augments and amplifies the impact of qualitative research.
The Anno-REP Pilot
The Anno-REP Pilot provides legal scholars an opportunity to pioneer the use of a new tool that facilitates the employment of ATI, a cutting-edge approach to enhancing the transparency of qualitative research. The pilot also offers QDR an opportunity to assess the effectiveness of the tool and learn more about the uses and benefits of ATI. Specifically, Anno-REP “piloteers” will (1) use the Anno-REP tool to annotate a manuscript they are currently writing and plan to submit to a journal, and (2) evaluate the use of the tool by one other piloteer. All piloteers will gather at a workshop on April 28-29, 2022 in New York City to evaluate the annotated manuscripts and consider how to improve the Anno-REP tool.
The pilot proposals have been selected and all applicants have been notified.
Criteria for Selection
Our goal is to select a set of manuscripts that, when annotated, will help QDR to identify the different ways in which ATI contributes to academic research and publishing; to better understand the challenges that arise in using ATI; and to evaluate to what degree Anno-REP addresses some of those challenges. Accordingly, we will seek to identify proposals / manuscripts that:
- collectively employ multiple analytic approaches, use diverse types of qualitative data, and examine various world regions and temporal periods;
- seem likely to be significantly enriched through the use of ATI;
- present annotation difficulties that QDR would like to figure out how to address;
- are methodologically sound, clear, complete, and convincing.
Detailed Timeline
Below we offer a detailed timeline for this project. QDR personnel will be available throughout the project to assist authors and answer their questions by email, phone, and Zoom.
- Submission of Manuscript (April 4, 2022): Each author whose proposal is accepted for inclusion in the ATI / Anno-REP Pilot is expected to complete their manuscript (of a typical length for published research in their field), annotate it with Anno-REP, and submit the annotated manuscript and underlying data sources to QDR by April 4, 2021 (please note - this is a firm deadline).
- Logbook / Questionnaire (April 4, 2022): If they consent to do so, authors will also be asked to keep an “ATI / Anno-REP logbook” as they annotate their manuscript, and complete a brief questionnaire chronicling their experiences using ATI / Anno-REP.
- Pre-print (April 11, 2022): QDR will archive authors’ data sources and post the annotated manuscripts to a pre-print server (or on a password protected page on the QDR website if the author prefers) by April 11, 2022.
- Review of Annotations (April 25, 2022): QDR will create author pairs (matching scholars who work on similar substantive topics) to evaluate each other’s annotated manuscript. Once the manuscripts are posted, QDR will share each project with the relevant reviewer. Reviewers will evaluate the use of ATI (not the manuscript itself), using a rubric QDR has created, by April 25, 2022.
- Workshop in NYC (April 28-29, 2022): Reviews will be shared with authors immediately, for consideration in the lead-up to the April 28-29, 2022 workshop. We anticipate offering a hybrid workshop with most attendees present in NYC, but with an option for remote participation (e.g. via Zoom).
Only scholars involved in the ATI / Anno-REP Pilot (and QDR personnel) will have access to the annotated manuscripts in advance of the workshop. With authors’ permission, QDR will make the projects public on the QDR website following the workshop. The ATI Data Supplement comprising the data underpinning the project (which will be citable with a digital object identifier [DOI]) will remain archived with QDR, and will be made accessible to registered QDR users after the workshop, with authors’ permission.
QDR strongly encourages authors to include ATI annotations in the eventual formal publication of their manuscript, and looks forward to working and supporting authors through the publication process.
The author(s) of each manuscript will receive an honorarium of $1,500. (Co-authors may share this honorarium in whatever way they see fit.)
Scholars interested in submitting a proposal are encouraged to contact QDR (qdr@syr.edu) to discuss their envisioned manuscripts and annotations and to ask any questions they may have.
Please do not hesitate to pass on word about this project to other faculty and graduate students whom you think might be interested
FAQs
What is ATI?
ATI is a new approach to openness in qualitative and multi-method research that empowers scholars to enhance their published work through “open annotation,” linking particular passages in the text to an analytic note, excerpt, underlying data source, and full citation.
Is ATI the only technique that can be used to make more transparent research that employs qualitative data and research methods?
No, ATI is not the only way to achieve research transparency in, and share data associated with, qualitative research, nor is it suitable for all types of qualitative scholarship. That said, we think ATI is a promising approach to openness in qualitative and multi-method research, and hope that the Anno-REP tool encourages more people to try out ATI!
Annotations can have four elements – an analytic note, one or more excerpts, a link to one or more data sources, and one or more citations. I understand an annotation only needs to have one of these elements, but if I want to include more than one or all of them in a particular annotation, in what order should I include them?
There is some flexibility in the ordering; the most important thing is for you to be intentional, i.e., to order the elements in the way you think will best assist the reader. All else equal, we suggest this order: analytic note, excerpt, source, citation.
All of my data sources are confidential and/or under copyright. Can I still use ATI and submit a proposal for the ATI / AnnoREP Pilot?
Yes. Please be sure to note in your proposal any difficulties you foresee with sharing the data that underlie your manuscript, and to describe how those difficulties may affect your use of ATI. Even if you cannot share all of the data arising from your research, providing excerpts from data sources that are under constraint, and using analytic notes, can enhance your manuscript. QDR is happy to answer questions you have in this regard; we also offer some guidance on the QDR site regarding sharing data generated through human participants research, and help in terms of sharing data under copyright as part of a self-guided on-line course we created on managing and sharing qualitative data.
My work is more theoretical than empirical. Can I still submit a proposal?
For the purposes of this pilot, we are seeking proposals for empirical legal scholarship manuscripts. We hasten to add, however, that some more theoretical work can benefit from using ATI; for instance, ATI might very effectively enhance a more theoretical manuscript that relies heavily on close textual analysis.
I am a quantitative researcher. Can I participate?
Maybe. ATI is designed for qualitative research. If you do multi-method work and your article contains significant qualitative portions based on original research, we would be happy to receive a proposal from you.
I want to annotate an article that has already been published. Can I still participate in the ATI / Anno-REP Pilot?
We are most interested in proposals from scholars who will be writing and annotating with Anno-REP simultaneously. Separate from the pilot, QDR would be very happy to help you to annotate a published article using ATI and Anno-REP at any time – and we encourage you to retrospectively annotate your work in this way!
If my proposal is selected for the ATI / AnnoREP Pilot, can I submit the manuscript to a journal between selection in January 2022 and the workshop in April 2022?
Yes. Your participation in the pilot and your scholarly publishing pursuits are independent. You should feel free to submit your manuscript to the journal of your choice as soon as you are ready! It would be best, however, if the annotations you create as part of the ATI / Anno-REP Pilot could be added to your manuscript before it is published so, for instance, the article can include a data availability statement .
If I want the annotations to be considered as part of the review process when I submit my manuscript to a journal, can QDR help me with a private version of the annotations?
Yes, QDR would be happy to help you with this.
Once my article is published, will QDR help me to include the annotations with my article?
Yes -- we will make our best efforts to help. If the journal has any objections, we can make public the pre-print version of the article for demonstration and learning purposes.
If my proposal to participate in the ATI / Anno-REP Pilot is rejected, can I still annotate my manuscript anyway?
Yes, absolutely! QDR would be glad to give you advance access to the Anno-REP prototype so you can do so.
Do I need the permission of an editor or journal to participate?
No, but you might need permission to include a link to your ATI data supplement (i.e., the amalgamation of the data sources that underlie your manuscript, stored on QDR) included in your article. We are happy to help with this process.