February 2022

Acknowledgments

The ABDC would like to thank the JQL Review Steering Group for preparing these recommendations about the frequency, methodology and scope of the JQL Review — Professor Andrew O’Neil (BARDsNet Chair), Professor Kim Langfield-Smith (ANZ Advisor), Professor James Morley (Expert Panel Advisor), Professor Julie-Anne Tarr (Expert Panel Advisor), Professor Peter Robertson (ABDC Executive Committee Advisor) and Professor Geoffrey Wood (International Advisor).

Introduction

The recommendations in this document are made by the JQL Review Steering Group, which was convened to review stakeholder submissions made in response to the June 2021 Consultation Paper, ABDC Journal Quality List: Review of Frequency, Methodology and Scope.

The Steering Group was comprised of domestic and international experts with recognised expertise in business disciplines and journal rankings.

The following stakeholders were invited to make submissions on the topics covered in the Consultation Paper:

• Academics with formal affiliations to universities in Australia, New Zealand, and internationally that have a legitimate interest in the disciplines covered by the ABDC list
• Schools or faculties that have a legitimate interest in the disciplines covered by the ABDC list
• Publishers
• Industry and professional associations.

The consultation period ran from 18 June to 18 August with 53 submissions received from stakeholders.

1.    Value and purpose of the JQL

Consultation submissions and Steering Group deliberations note several benefits of the ABDC JQL:

• Global currency — the list is globally recognised, which is an endorsement of its rigour and standing.
  • Institutional recognition — the established nature of the list provides trusted guidance to ANZ universities and business researchers in recognising quality business research.
  • Methodology — the combined use of peer review and metrics makes the JQL a valued resource for ANZ researchers and business schools to identify quality outlets for publication.

Recommendation

The ABDC JQL should be retained because its benefits to Australian and New Zealand business schools and researchers outweigh the costs of undertaking list reviews. Future communications from the ABDC should continue to underscore the value of the list as a guide to quality business research.

2.    Frequency of review

The five-year interval is strongly endorsed as the presumption that should guide the timing of comprehensive rankings reviews, noting sector fatigue and the resources required to manage the review process. Submissions and Steering Group deliberations support more regular, partial reviews that are tightly framed to address issues such as predatory journals and anomalies. Changes to the ANZSRC should be addressed through a partial review to map listed titles to the new FOR codes.

Recommendation

The ABDC should address field of research code changes through a partial review in 2022. This review will also (like the 2013 and 2016 partial reviews) focus on the inclusion of new journals; removal of very low quality (including potentially predatory) journals; and correction of any factual errors. The next comprehensive rankings review should be undertaken in 2024.

3.    Conduct of review

The existing process of forming Expert Panels to oversee the JQL review is generally thought to work well. In the appointment of future panels, the ABDC should:

• Ensure diversity along gender, culture, geography, and career stage dimensions to promote the robustness of panel recommendations.
  • Accord greater transparency to panel operations by reporting on how conflicts of interest were managed.
  • Mitigate panel workloads by implementing strategies to reduce frivolous submissions, e.g., by encouraging submissions made on behalf of stakeholder groups jointly as opposed to individuals within those groups.
  • Encourage panels to consult with industry and policy participants where relevant.

Recommendation

The ABDC should continue to utilise Expert Panels to conduct JQL reviews, considering the points raised above regarding diversity, transparency, panel workload, and industry consultation.

4.    Factors that determine journal quality

The ABDC JQL is distinctive because it is a peer assessment of journal quality informed by relevant metrics. While Expert Panels should be informed by quantitative measures of journal quality, a formulaic adoption of metrics is counter to the purpose of the list.

The Steering Group believes that the ABDC should publish guidance about the role of both metrics and peer judgment in assessments of journal quality. This guidance could identify examples of quantitative and qualitative variables considered by Expert Panels in exercising judgment about journal quality.

Recommendation

Communications from the ABDC should emphasise that the purpose of Expert Panels is to exercise judgment about journal quality, guided by available metrics and impact factors where these are informative.

5.    JQL coverage

A key question for stakeholders and the Steering Group was the extent to which the JQL should include interdisciplinary titles of broad relevance to business researchers in foundation disciplines such as sociology and psychology.

Caution should be exercised in altering the business element test because it has been applied diligently in previous reviews. Liberalisation of this test could result in a significant increase in listed titles, which is not desirable.

Where practitioner focussed journals are listed it is with reference to the ranking criteria applied to all journals, noting that the list is designed to guide the evaluation of research according to ANZ frameworks.

Recommendation

The ABDC should provide a statement about the inclusion of interdisciplinary journals in the JQL to clarify that the role of Expert Panels is to take a common sense, not mechanistic, approach to their inclusion guided by the ‘substantive business element’1 test. The statement should also reiterate that the JQL is designed to indicatively, not comprehensively, include interdisciplinary journals. Equally, exclusion of journals should not be used to discourage interdisciplinary research.

6.    Consideration of other lists

Stakeholder submissions and Steering Group deliberations agreed that Expert Panels should take a contextual approach in referring to other journal lists. Lists such as CABs, FT and Dallas can provide useful guidance for Expert Panels, but consistency with these lists is not a goal given that the ABDC JQL’s primary purpose is to provide an ANZ lens on journal quality. The ANZ lens on journal quality does not, however, mean automatically weighting local journals.

The JQL is distinctive because it recognises the importance of ANZ business research and is composed and closely supervised by ABDC-member business schools. For these reasons, support did not exist for merging the JQL with a comparable list.

Recommendation

Expert Panels should continue to take a common sense and contextual approach when referring to other journal lists, noting that consistency with these lists, while preferable, is not a goal.

1 Evidenced by >50% of articles over three years written by business faculty, or >50% of articles over three years being of a business nature.

7.    Thresholds for journal rankings

There was broad agreement that the percentage thresholds are a useful constraint for Expert Panels in ranking journals. The ABDC should not change the current thresholds unless there is good evidence or justification to do so.

It is noted in the report of the 2019 rankings exercise that there is a stickiness in journal rankings across successive iterations of the list. Over time there has been an increase in the number of journals ranked A and A*, suggesting that established journals are improving over time. Stickiness of rankings is challenged by the issue of due process for listed journals that may be past their prime, and management of journals with inflated rankings.

The Steering Group agreed that there may be justifications for variations in rankings across disciplines reinforcing the purpose of Expert Panels to exercise judgment and use the thresholds as a guide.

Recommendation

The current thresholds should be retained. The ABDC should support future Expert Panels to downgrade, or potentially remove, journals that no longer match the threshold criteria. The ABDC should consider adding selectivity as a criterion for A and A* journals.