Course Description: Analyzing textual data can be approached inductively or deductively, depending on the selected methodological approach of the research project. In this workshop, we will discuss and undertake hands-on text analysis exercises. Top down or a more confirmatory approach to text analysis will cover topics such as: dictionary application (e.g., Zavyalova, et al., 2016 & LIWC dictionary; Short et al., 2009); dictionary creation (Franco, Alexander, & Smith, working paper; Short et al., 2010); template utilization (Crabtree & Miller, 1999; King, 2004); and collocation analysis (Gephart, 1997). Students will be working with textual data to explore these techniques. Bottom up or more exploratory approach will include an in vivo, manual coding exercise and a demonstration of coding techniques using computer aided qualitative data analysis software (CAQDAS). No prior knowledge of software or text analysis is required.

Advanced Multilevel Analysis

We will start with an overview of the “basics” underlying multilevel analyses.  Doing so is to simply refresh memories because participants should have a solid grounding in multilevel analysis to benefit from this advanced short course.  For example, if you took my CARMA at Adelaide short-course in March 2017, you are in good shape.  Or if you have conducted multilevel analysis in your own research, you should be just fine.  After reviewing the basics, we will spend some time understanding how those basics may be viewed from a structural equation modeling (SEM) perspective.  Time will be spent on how we may view measurement models and SEM path models within a multilevel framework.  Subsequently, we will turn to conducting SEM multilevel analysis starting with models justifying aggregation of latent variables upward (i.e., computing ICC1 and ICC2 but using latent variables instead of observed variables).  From that point forward, we will analyze increasingly complex path models.  Through the latter, you will be exposed to testing cross-level interactions among the latent variables, and how to undertake tests for mediation.  I follow a format where I conduct an analysis, and then you are given an opportunity to conduct your own.  I will supply example databases, but you are also encouraged to use your own.  I use the Mplus SEM software package in the short-course.

Course Description: This short course will begin with a brief review of correlations and hypothesis testing. Next the basics of linear regression with a single predictor will be introduced, including consideration of OLS estimation and associated statistical testing (e.g. t-test, F-test, confidence interval, statistical power). These basics will be introduced using a continuous predictor, and then dummy variables for categorical variables will be covered. Finally, extension of the above topics to the case of two predictors will be covered. For most topics, examples and assignments will be provided.

Discoveries We Anticipate

I and others have written about the various rules, some written some not, that determine what does and doesn’t get published.  These formulas do serve a purpose, but they also suppress creativity and innovation.  In this talk, I will describe three papers that explore alternatives to the models and designs that are typical of the organizational sciences.  First, I will describe Cortina, Koehler, Keeler, and Nielsen (under review) in which we tease apart the various forms of restricted variance interaction.  Next, I will describe Cortina and Markell (under review) in which we develop the theoretical and mathematical underpinnings of Endogenous Moderator Models.  Finally, I will describe Koehler and Cortina (under review) in which we investigate the different forms that replication can take and demonstrate that our field uses only a small number of them.

Measurement, Research Design and Effect Size in an Era of Increased Data Analysis Sophistication

I will describe the advances made in quantitative analyses during the course of the last several decades and, in particular, during the history of CARMA.  I will then describe three areas which should receive more attention.  First, we have not paid much attention to the construct validity or psychometric quality of our measures.  Second, I think research design needs more attention, especially the timing of measurements in longitudinal research.  Finally, I think we should make greater efforts to communicate the meaning and impact of our research to potential users.  In each of these areas, I will suggest some efforts that should be undertaken to address these issues.

Model Evaluation in Structural Equation Research: Challenges and Solutions

Structural equation modeling (SEM) serves as one of the most important advances in the social sciences in the past 40 years.  It has become widely used in organizational research, with a recent review of six topic management journals indicating that 237 articles based on latent variable path models were published between 2001-2014.   A key part of the use of SEM involves researchers evaluating their models, because conclusions about hypotheses from misspecified models can not be trusted.   Practices for model evaluation have evolved over time, suffer from limitations, and new techniques are being developed at a rapid pace.  This lecture will summarize key steps in model evaluation, some limitations of current approaches, and solutions with promise that may be of value for organizational researchers.

Techniques to Build Trustworthiness in Coding using Computer Aided Qualitative Data Analysis Software

Trustworthiness is the established approach to evaluate qualitative research. Many dimensions of trustworthiness and activities to improve trustworthiness have been incorporated into the methods of research in top management and organizational journals. One area of inductive qualitative research that has not been fully fleshed out in terms of aspects of trustworthiness is coding activity. This is surprising given advancements in computer aided qualitative data analysis software (CAQDAS) to enhance trustworthiness. In this workshop, I identify procedures to enhance trustworthiness in coding and identify supporting software techniques and inductively analyze several letters to shareholders from four companies.

Advancing rigor and relevance: A framework for replication and reproducibility

Replication in science has received a great deal of attention over the years.  The organizational sciences, though, have seen a particularly strong increase in the discussion of replication. At the same time, there remains skepticism about the existence of a replication crisis. The claims of skeptics fall into three categories.  First, replication is quite common, so there is no pressing need for more of it.  Second, replication studies are generally quite flattering to previous results.  Third, where results fail to replicate, the explanation is that the replication study differed from the original. Arguably, much of the skepticism regarding the need for more replication stems from a lack of understanding of the different forms that replication can take, the prevalence (or lack thereof) of many of these forms, and the objectives that are met by the less common forms.The current presentation will provide an overview of prior definitions of replication to derive one integrative definition and a distinction from reproducibility and generalizability.  I will then describe two overarching dimensions along which both replication and reproducibility studies vary.  Combining these dimensions, I describe some of the more common forms of replicability and reproducibility (as represented by specific combinations of the two underlying dimensions) and the methodological problems that they target.  I then describe some of the less common forms of replicability and reproducibility and the (different) methodological problems that they target.  Next, I dig deeper into one of the most important combinations, the independent constructive replication, in order to distinguish the more useful from the less useful types of constructive studies.  Audience members should leave the presentation with a well-founded understanding of the differences between replication and reproducibility and the different forms that both can take in different study designs.

Doing the “Right” Thing Rather Than the “Popular” Thing: A Statistics and Methods Perspective

We will review some of the more common bad practices from a statistics and methods perspective with a focus on why it is bad and on what alternatives may be engaged in to avoid it.  A large part of this master class is based on the speaker’s work on statistical and methodological myths and urban legends.  However, it is also based on recent publications co-authored with Herman Aguinis, and with Jose Cortina and two of his PhD students, Jennifer Green and Kathleen Keeler.

Demonstrating rigor in qualitative research analysis

As a qualitative researcher, chronicling and justifying how one moves from data to theory remains a dilemma. The ‘Gioia methodology’ (Gioia et al. 2013) is gaining popularity as a solution to this dilemma. In this master class, the steps in the Gioia methodology will be introduced and critically examined. It will be argued that while the Gioia methodology provides a process for ordering and conceptually labeling data, it does not provide qualitative researchers with the route to a theoretical contribution. Problematization and contextualization will be advocated as the basis for theoretical insight, rather than the assumptions of the Gioia methodology, which can be traced to classical grounded theory. The aim of the workshop is therefore to instigate a critical reflection of how qualitative researchers go about fashioning a novel theoretical contribution from their fieldwork.

The Introduction to Structural Equation Modeling (SEM) workshop will provide participants with (a) the basic concepts behind SEM and the assumptions involved, (b) an overview of various commonly used models including measurement model for confirmatory factor analysis, path model and full structural equation model with latent variables, (c) comparisons of these models with traditional analytical approaches such as exploratory factor analysis and regression, and (d) examples of applications of SEM in answering contemporary organizational research questions involving mediation, moderation, moderated mediation, measurement invariance, longitudinal data and multilevel data and theory.

The purpose of this workshop is to introduce researchers to the underlying tenets of the grounded theory approach and to get them started in designing and conducting a grounded theory study. In the workshop, I will provide an overview of the following three major topics: (a) Understanding the approach and different methodological traditions, (b)  core design characteristics of a grounded theory study, (i.e., purposive sampling, iterative data collection and analysis, triangulation and use of different sources and data), and (c) analyzing data following the grounded theory method (i.e., different approaches to coding, constant comparison, memoing, triangulation, theoretical saturation, etc.). We will use some concrete examples throughout the workshop to provide hands-on advice and practical guidance.

The case study in management research has been heavily influenced by Eisenhardt and Yin, both of whom started publishing on the case study in the 1980s. They succeeded in legitimising this research strategy and sparking what has been called a ‘renaissance’ in case study research (Ketokivi and Choi 2014). But while the type of ‘modernist’ case study advocated by Eisenhardt and Yin is still very influential in management journals, alternatives are being developed and are gaining ground. In this workshop, we critically evaluate the dominant templates for case study research, as well as these emerging alternatives. By the end of the workshop, participants will have gained a clear understanding of different positions on the case study, and will have obtained insights into how recent methodological advances can inform their own research. Concrete examples of case studies will be provided and will form the basis for discussion.

This workshop will explore research designs where multilevel modeling is required. Starting with groups and individuals, we will explore questions about aggregation, cross-level effects, and multiple group membership. These questions will then be extended to further levels of anlaysis such as organisations and countries.

The next part of the workshop will consider longitudinal measurement from a multilevel perspective. Growth curves and other change models will be reviewed and we will conclude by considering intensive longitudinal data.

Mplus code will be made available for the examples we cover, but MPlus is not essential for the workshop.