RESQUE: The Research Quality Evaluation scheme for psychological research

News

See all news here.


  • 2024-03-21: Einstein Foundation Award
    Anne Gärtner received the Einstein Foundation Early Career Award 2023 for the project on Responsible Research Assessment. The project is based on RESQUE and aims to further…
  • 2024-02-26: New indicators for theory-guided research developed
    A working group consisting of Nele Freyer, Jens Lange, Daniel Leising, Philipp Musfeld, and Felix Schönbrodt developed a set of indicators that allows to evaluate…
  • 2023-12-12: RESQUE website is live!
    We launched the RESQUE website. All material will be collected here, including news and information about case studies, evaluation projects, new indicator packs, etc.
  • 2023-09-04: Preprint published by Franka Etzel on an evaluation study
    63 papers nominated by 21 participating researchers from personality and social psychology were rated by external raters (i.e., not the authors themselves) with the…
No matching items

The Research Quality Evaluation (RESQUE) framework provides recommendations for a responsible research assessment that does not rely on flawed metrics such as the journal impact factor or the h-index.

In alignment with the principles of CoARA, this approach acknowledges diverse academic contributions, prioritizes the quality of work rather than its volume, and integrates qualitative peer assessment with the responsible use of quantitative indicators. All proposed indicators are open-source and reproducible.

What is RESQUE? Three modular elements

1. Evaluated Indicators

Indicators Image

✅ A set of evaluated indicators for aspects of research quality.

👉🏼 "Buffett table" approach: Escape h-index and JIF by choosing valid indicators that suit your research assessment scenario.

2. Process Suggestion

Process Image

✅ A suggestion for a two-stage process and weights for indicators.

👉🏼 Change the importance of indicators depending on your use case.

3. Tools Set

Tools Image

✅ A set of tools that can be used (optionally):

  • A Collector App where applicants can enter their data
  • Profile creation for CVs
  • Interactive comparative overview for committees

👉🏼 Open source, free to use, privacy aware. Of course you can also include the selected indicators in your existing recruitment system.

Primarily designed to assist hiring and tenure committees, it emphasizes that the indicators and algorithmic methods serve as tools to support, not replace, decision-making processes. By automating the generation of relevant candidate information, this approach enhances the effectiveness of human expertise in evaluating potential hires and tenure candidates.

RESQUE aims to provide objective quality and impact indicators for three types of research outputs:

  1. publications of empirical studies,
  2. published data sets and
  3. research software.
Important

The RESQUE framework is currently in development and the indicators are subject to change. Currently, only indicators for publications are available in a beta version.

The assessment scheme is primarily developed for the field of psychology, but might be easily transferred to neighbouring empirical scientific fields.

Built by the community for the community

The SCOPE Framework for Research Evaluation proposes the principle “Evaluate with the evaluated”, advocating for a co-creation process within the communities that are being evaluated. RESQUE is developed in that spirit. We started with four target papers that elicited 40 publihsed commentaries, which led to a major revision. (Many critical commenters now are in the developer team.) We had workshops with most field-specific groups within the DGPs, which also led to substantial revisions and the development of discipline-specific expansion packs.

Resources

☑️ Rating schemes: Core set

With the goal to make data entry for applicants and hiring committees as easy and frictionless as possible, we developed the RESQUE Collector App that allows to provide information for your best research outputs and to export the data in a JSON format.

We plan to provide three rating schemes for …

  • RESQUE-Pubs (Publications, preprints, book chapters, etc.): ✅ v0.4.0, ready for use (but still likely to change).
  • RESQUE-Data (Data sets): ❌ under development, not implemented in Collector App yet
  • RESQUE-Software (Research software): ❌ under development, not implemented in Collector App yet

☑︎ Rating schemes: Expansion packs

We aim to develop and collect disciplinary expansion packs (EPs) with specific indicators that are relevant for a subfield. If you plan to provide an expansion pack, please get in contact with us (we will help to implement that in the web form).

Such EPs ideally are contributed from a legit academic community and stem from a consensus process. Before including EPs in this project, we do a minimal review for suitability.

The subdivisions of the German Psychological Society have been asked to discuss the need for discipline-specific indicators.

Available Expansion Packs:

📊 The RESQUE Research Profile

We provide R scripts that …

  1. load and aggregate multiple json files with RESQUE ratings
  2. enrich the data with other sources of information (e.g. citation counts & normalized citation counts from OpenAlex, impact metrics from BIP!)
  3. provide candidate profiles that can be requested by hiring and tenure committees, or used by candidates as an attachment to their CV or website.

An interactive dashboard allows to compare multiple candidates and to zoom into the profiles of specific candidates.

For more information, see here.

Partners and supporters of RESQUE