RESQUE Profile for Max Mustermann Demoprofile

The “fingerprint” of how research is conducted, when only the best work is submitted.
Analysis from 2024-10-08
Academic Age: 14. Analysis based on 10 publications.

Note

This is a preview which shows some visual summaries of the RESQUE indicators. Not all indicators have been covered yet, and things might change substantially. No bonus points have been assigned to the theory indicators yet, and also not to some other indicators of sample characteristics.

Some parts of this profile are purely descriptive. For example, it is summarized the type of participant samples, or whether researchers predominantly work with psychophysiological data or rather focus on questionnaire studies.

Other parts contain an evaluative aspect: For example, research that is reproducible, which allows independent auditing because it provides open data and scripts is, ceteris paribus, better than research that does not have these aspects. Research outputs with these objective quality criteria of methodological rigor can gain “bonus points” which are summed across all provided research outputs and contribute to the Rigor Profile Overview.

We took care not to systematically disadvantage certain fields or research styles. Generally, the rigor score is a relative score, computed as “percentage of maximal points” (POMP) score across all indicators that are applicable. For any indicator, one can choose the option “not applicable” if an indicator principally cannot be attained by a research output. The points of such non-applicable indicators are removed from the maximum points and therefore do not lower the computed relative rigor score. However, in order to prevent gaming of this scheme, any “not applicable” claim needs to be justified. Only when the justification is accepted by the committee, the point is removed. With no or insufficient justification, in contrast, the indicator is set to “not available” (=0 points) and the maximum points are not adjusted.

Submitted research outputs

10 research outputs have been submitted.

The 10 publications had the following methodological type:

Type of method # papers
Computational 5
Empirical Quantitative 4
Meta Analysis 2
Other Method 1
Theoretical 3

Team science in publications?

10 out of 10 submitted publications could be automatically retrieved with OpenAlex.

Team category Frequency %
Single authored 0 0%
Small team (<= 5 co-authors) 8 80%
Large team (6-15 co-authors) 0 0%
Big Team (>= 16 co-authors) 2 20%

Types of research data

Data type # of papers
Behavioral 1
Other Type 1
Other 0
Content Data 3
Questionnaire Selfreport 1

⤷ Types of behavioral data

Data type # of papers
Performance 1

Types of samples

Type of population/sample and representativeness for the 3 papers with own new data collection (“Other” excluded):

Contributorship profile (CRediT roles)

Based on 10 submitted publications, this is the self-reported contributorship profile:

Indicators of Research Transparency and Reproducibility

Overview of open research practices

Open Data Available Preregistration available Open Material available Open Code available Correctness of computational results has been independently verified

Red = not available; shades of green = available to increasing degree; grey = not applicable

Rigor profile overview

The relative rigor score (RRS) is computed as a “percentage of maximal points” (POMP) score of multiple indicators. The indicators are grouped into four categories: Open Data, Preregistration, Reproducible Code & Verification, and Open Materials. Indicators that are flagged as “not applicable” are removed from the maximum points and therefore do not lower the RRS.

The general philosophy of RESQUE is: It doesn’t matter so much what kind of research you do - but when you do it, you should do it in a high quality. The radar chart with the Relative Rigor Score helps you to see how many quality indicators have been fulfilled in multiple areas of methodological rigor.

  • The width of each sector corresponds to the maximal number of rigor points one could gain. If many indicators are flagged as “not applicable”, then the maximal points get reduced and the sector gets more narrow.
  • The colored part of each sector shows the achieved rigor points. An entirely grey sector indicates that no rigor points could be awarded at all.
  • The quality indicators measure both the presence of a practice (e.g., is Open Data available?) and the quality of the practice (e.g., is does it have a codebook? Does have a persistent identifier?). Hence, even if the pie charts in the table above show the presence, a lack of quality indicators can lead to a low rigor score.

Internationality and Interdisciplinarity

The analysis is only based on the submitted publications (not the entire publication list) of the applicant. Publication and co-author data is retrieved from the OpenAlex data base. Note that preprints are not indexed by OpenAlex and therefore do not contribute to this analysis.

  • Internationality: All co-authors are retrieved from OpenAlex with their current affiliation. The index is measured by Pielou’s Evenness Index (Pielou 1966) of the country codes of all co-authors. It considers the 10 most frequent country codes.
  • Interdisciplinarity is measured by the Evenness Index of the fields (as classified by OpenAlex) of the publications. It considers the 6 most frequent fields.

The evenness indexes are normalized to a scale from 0 (no diversity, everything is one category) to 1 (maximum diversity, all categories are equally represented). It is computed as a normalized Shannon entropy.

Internationality
Interdisciplinarity
Only within country co-authors
Broad coauthor network from many countries
Single discipline
Many disciplines
49 unique identifiable co-authors:
  • 49% from 7 international countries
  • 51% from the same country
2 primary fields:
  • Psychology (8)
  • Decision Sciences (2)
Co-authors' Country Code # of Co-authors
DE 25
US 10
NL 6
AT 2
AU 2
CH 2
IT 1
SE 1
The main subfields are (multiple categories per paper are possible):
Subfield # of papers
Experimental and Cognitive Psychology 6
Clinical Psychology 5
Applied Psychology 4
Social Psychology 2
Sociology and Political Science 2
Statistics and Probability 2