Speaker: Prof. Dr. Andrea Hildebrandt, Psychological Methods and Statistics, Department of Psychology, Faculty VI. – School of Medicine and Health Sciences, Carl von Ossietzky Universität Oldenburg (UOL)

Coordinator: Dr. Dorothe Poggel (HWK)

Members: 

• Prof. Dr. Stefan Debener, Neuropsychology, Department of Psychology, Carl von Ossietzky Universität Oldenburg
• Dr. Carsten Gießing, Biological Psychology, Department of Psychology, Carl von Ossietzky Universität Oldenburg
• MSc. Nadine Jacobsen, Neuropsychology, Department of Psychology, Carl von Ossietzky Universität Oldenburg and Open Science Speaker of the University 
• Dr. Daniel Kristanto, Psychological Methods and Statistics & Biological Psychology, Department of Psychology, Carl von Ossietzky Universität Oldenburg
• Dr. Cassie Short, Psychological Methods and Statistics, Department of Psychology, Carl von Ossietzky Universität Oldenburg
• Prof. Dr. Christiane Thiel, Biological Psychology, Department of Psychology, Carl von Ossietzky Universität Oldenburg

Duration: March 2023 – February 2026

Problem Statement

Meta-Science, the emerging field of research dedicated to study the scientific process itself, is in its momentum. Many of us believe that it has become a permanent and important discipline in itself, along with the open science movement it promotes. Meanwhile, researchers and funding agencies spanning all scientific disciplines are increasingly emphasizing the role of Meta-Science in diagnosing and curing the replication crisis in science. Procedures such as study preregistration, open materials and open data, and registered reports are meanwhile widely practiced. They are also promoted and anticipated by many high-ranked journals and academic institutions. However, certain procedures proposed by the open science movement are still far from being used regularly. Methodological solutions that aim to add transparency to the black-box created by researchers’ degrees of freedom are only implemented by small communities characterized by extensive expertise in programming and data science. These communities feel urged to create user-friendly software solutions allowing to explore and address the multitude of choices in the scientific process, exclusive of a formal training in scientific programming and big data analyses. Furthermore, today meta-scientists engage in implementing systematic expertise crowdsourcing strategies to make domain specific, scientifically sound (defensible) choices in the study process explicit and easily accessible by entire scientist communities. 

To describe the multitude of researchers’ methodological choices, Gelman and Loken (2013) coined the term “garden of forking paths”. In every step of the study planning, data processing and analysis workflow, multiple defensible decisions and potential operations are available as choices. The established approach in science to date is to select one specific workflow from which results are reported and conclusions of the phenomenon studied are made. However, a multiple comparison problem is implicit to this approach, even if only one constructed dataset is used for statistical inferences. The reason is that theoretically a large variety of workflows are defensible, yet researchers do not correct their hypothesis tests to account for all theoretically possible comparisons that they did not explicitly carry out.   

More recently, the term multiverse has been proposed for the above elaborated problem (e.g., Steegen et al., 2016) to underline the high dimensionality of the garden of forking paths. Today, meta-scientists differentiate three domains of multiverse in the scientific process. Design multiverse refers to the study design specification, data multiverse to data processing and parametrization choices, and model multiverse to the statistical analyses options available to address specific research questions and/or hypotheses. The resulting multiverse is immense, domain specific and highly dynamic. This is because, for example in the neuroimaging field, methodological solutions are improved upon and advanced with huge pace. For small teams of scientists, such as single laboratories, it is thus impossible to gain a comprehensive map of the multiverse of defensible choices at a given moment in time. We need large collaborating communities (Wacker, 2017) and dynamic expertise crowdsourcing platforms to continuously map the multiverse in specific fields. 

Our study group aims to map and visualize the multiverse in two neuroimaging domains: (1) Stationary and mobile electroencephalography (EEG) applications in cognitive and personality neuroscience and (2) functional magnetic resonance imaging (fMRI) for network neuroscience approaches to individual differences in cognitive abilities. Furthermore, we aim to develop and promote the use of software solutions for a more easily applicable multiverse research in these domains. The focus group is tightly linked to the METEOR project embedded into the DFG Priority Program META-REP (“A Meta-scientific Program to Analyse and Optimise Replicability in the Behavioral, Social, and Cognitive Sciences”, SPP 2317) and to the entire program. We also collaborate with the DFG funded CoScience – The EEG Personality Project. Furthermore, our activities are open to the members of the DFG funded Research Training Group “Neuromodulation of Motor and Cognitive Function in Brain Health and disease” (RTG 2783). 

At the HWK, the Study Group contributes to the institute's focus on the quality of academic research.

Planned Activities

• We will carry out community workshops and launch interlinked crowdsourcing expertise platforms to map the multiverse in the domain of stationary and mobile EEG research of individual differences in cognition and personality, as well as network neuroscience fMRI approaches to individual differences in cognition.
• Aiming to make the outcome of the community discussions a collective good, we will summarize them along with the agreements and remaining controversies regarding the defensible choices in the multiverse in the form of opinion papers and guidelines for the above fields of interest.
• As supporting tools to accompany such guidelines, we will create open source and extendable visualizing tools which will allow researchers to navigate through the community agreed map of the multiverse.
• Within the DFG project METEOR, we are about to create easily adaptable analyses pipelines driven by machine learning with the aim of computationally managing the multiverse. These pipelines will be promoted by the study group.  
• On a more general level, we will carry out a series of educational activities (launching interactive and open learning tools for multiverse analyses, hands-on workshops) in collaboration with the Open Science Interest Group (OSIG) at the Department of Psychology, UOL. Thus, we will promote knowledge on multiverse analyses in the broader cognitive neuroscience community, specifically targeting early career scientists of the region Oldenburg/Bremen, potentially extending to further Institutions in North Germany and Groningen. 

Work Plan for the First Year

We will design three interlinked expertise crowdsourcing platforms (stationary EEG; mobile EEG; fMRI) and invite the community to share their expertise with respect to the scientific soundness of a multitude of decision options in the domain of design, data and model multiverse. We will focus on decisions hitherto collected from the literature and the community) that are invariant across a series of research questions typically addressed in the above mentioned neuroimaging fields. We will then analyze and summarize the collected expertise, specifically with the goal of pointing to controversies within the community with respect to defensible design, data processing and analytic choices. These controversies will then be targeted and potentially solved in focus groups, organized as in person workshops (separately for the EEG and fMRI field) planned to be carried out early 2024.     

Work Plan for the Second Year

We will carry out a 3-day workshop (one day per neuroimaging domain, with potential cross-domain participation) at the HWK. The goal is to bring together a group of experienced and early career scientists for each neuroimaging domain to discuss and potentially solve the controversies with respect to which design, data processing and analyses choices in the respective domain remain defensible. The outcomes of the workshops will be summarized in opinion papers and community guidelines, along with supporting tools. Furthermore, the focus group will collaborate with OSIG and the RTG 2783 and create open educational resources and organize hands-on workshops to promote multiverse analyses skills of early career scientists. 

Work Plan for the Third Year

The activities in the first and second year specifically target the data and model multiverse. In the last year we will more closely address the problem of design multiverse. There is evidence that many neuroimaging studies suffer from low reliability and reproducibility. A simple, however very expensive and in many cases non-practicable solution is to drastically increase the sample size of neuroimaging studies. It is currently unknown whether specific forking paths regarding design specification, data processing, and statistical analysis would improve the robustness and reliability of neuroimaging studies. This still needs to be systematically explored. Is it possible to identify forking paths best suited to small-sample studies? We aim to organize a workshop to address this prominent and crucial challenge in cognitive neuroscience, which is the problem of small samples. We will invite experts in statistics who engage in developing statistical tools for the analyses of small data to give a 2-day hands-on workshop. Above the hands-on part of this workshop, the goal will be to discuss applications in the domain of neuroimaging.  

References

Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. 

Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multi-verse analysis. Perspectives on Psychological Science, 11, 702–712.

Wacker, J. (2017). Increasing the reproducibility of science through close cooperation and forking path analysis. Frontiers in Psychology, 8, e1332.

Coordination: Dr. Dorothe Poggel, HWK

Members:
Petra Groß, Karin Dedek, Beate Grünberg, Christoph Lienau, Jens Christoffers, Arndt Meyer, Petra Bolte, Henrik Mouritsen, Ulrike Janssen-Bienhold, Karl-Wilhelm Koch, Christine Köppl, and Hans Gerd Nothwang - Carl von Ossietzky University Oldenburg, Herman Offerhaus - Optical Sciences Group, University of Twente, The Netherlands
Carsten Fallnich - Optische Technologien, Westfälische Wilhelms Universität Münster
Sonja Pyott - University Medical Center Groningen, The Netherlands
Kathrin Thedieck - European Medical School (EMS), Universities of Groningen and Oldenburg

Duration:  01. November 2016 bis 31. Oktober 2019/ 01. November 2016 – 31. October 2019

Content:
The working principle of the building blocks of life is one of the great questions that drive scientists of different disciplines alike. These fundamental questions are where biologists, chemists, and physicists meet and discuss. Examples of wide-ranging and interdisciplinary questions that we would like to develop into specific research question as a starting point for bi- and multilateral projects are:

•  How can photoreceptors “see” single photons?
•  How can the hair cells of the inner ear transmit sound with microsecond precision?
•  How are the multi-molecular complexes of gap junctions regulated?
•  How can the olfactory system distinguish between thousands of different smells?
•  How can birds detect small changes in the earth magnetic field to orient themselves during   their migratory flight?
•  How does the molecular machinery of the inner ear enable our ability to hear?
•  Which specific brain regions are involved in the hearing processes in the auditory brainstem?
•  Which protein-protein interaction partners are the basis for the above mentioned and for other mechanisms?
•  At the same time, there is a wide variety of analytic tools and, in particular, optical imaging methodology, available. Examples of the optical imaging techniques that are already available or that are currently under  investigation include:
•  Conventional and in particular home-built laser scanning confocal microscopes
•  Synthesis of novel and tailor-made fluorescent molecules
•  Light sheet microscopy
•  Coherent anti-Stokes Raman scattering (CARS) or stimulated Raman scattering (SRS) microscopy
•  Stimulated emission depletion (STED) microscopy
•  Scanning near-field optical microscopy (SNOM).
•  Ultrafast spectroscopy

Bringing together the disciplines of biology, chemistry, and physics holds a chance to discover truly new findings about the nature of life. Examples are the potential role of quantum effects and quantum coherences in the molecular biology of some sensory systems (e.g. magnetoreception in birds), and the molecular composition and regulation of electrical synapses.

Essentially the goal is to work out where we could apply which optical microscopy method. We aspire bridging the gap between our departments and to develop and apply new optical methods and techniques in order to answer prevailing biological questions. The goal of the Study Group is to establish bi- and multilateral co-operations, within which we apply new non-invasive, non-bleaching, and sub-diffraction-limit optical imaging techniques to highly sensitive neurobiological samples.

Planned Activities:

1. Kick-off meeting in combination with the DFG RTG (Graduiertenkolleg) “Molecular Basis of Sensory Biology”: November 2 – 4, 2016
2. Half-year workshop: One day at the HWK, after month 6
3. Interim workshops: one day each at the HWK, after month 12 and after month 1
4. Final study group workshop/ International Symposium with integrated spring school for PhDs on (nano-)optical microscopy in sensory systems

Goals:

1.  Identification of research questions and approaching these questions in joint projects
2.  Publications in peer-reviewed journals written jointly by two or more groups within the study group
3.  Strengthening of the cooperation between neuroscientists within the European Medical School (EMS) and the regional universities
4.  Institutionalization of a network of young female scientists in optical imaging and neuroscience
5.  Internationalization of the DFG RTG “Molecular Basis of Sensory Biology” and the RTG of the state of Lower  Saxony “Nano-Energy Research” by establishing contacts, e. g., during joint workshops and symposia.