Research Competencies
Currently, the validity requirements from an empirical perspective emphasizes generalizability, and the
primary data collection instrument is observation and measurement (Guba & Lincoln, 2005). Through observation, the focus only is on what can be seen; to focus on what another person could also access. To achieve objectivity, empirical researchers must remove most subjective experiences in order to articulate to others a mutual understanding of the object of study. This third-person observer perspective is the basis for external validity or generalizability, which “concerns inferences about the extent to which a causal relationship holds over variations in persons, settings, treatments, and outcomes” (Shadish, Cook & Campbell, 2002, p. 83). Often, threats to external validity are related to causal interactions from among features in experimental condition, including participants, treatment, research setting and results.
However, separating objective knowledge claims from subjective claims exposes limits of empiricism.
Understanding what results occurred is different from explaining why the results occurred. In qualitative studies, construct validity brings together empirical evidence and theoretical rationale to support the appropriateness of the inferences researchers make about the evidence. For example, in Dr. Glawewski’s Case-based Reasoning research group, the focus of our qualitative investigation was to reframe an understanding of the idiosyncrasies associated within a research setting and underscore the perspectives of the participants. In other words, emphasizing the specific instances of social practice can be “generalized to other similar situations” (Erickson, 1986, p. 130). The validity claim has less to do with a logical form of an argument but with a social idea the acceptability of a knowledge claim given a particular context. The focus from external validity to construct validity changes to whether the construct definition and the way it is measured is consistent across the particular context of a research study.
In the context of education research, my position as a researcher is shared in part with Dr. Ted Frick’s
explanation about the need for a philosophical educology (2015). Education research must not only be become precise as discipline with clearly and consistently defined terms, the justification for conducting research must be founded on rationale principles rather than “empirical fact” (pp. 6-7). For example, we do not, as an empiricist would presume, perceive objects and then define them but instead we define first and then perceive. That we are socialized to think in particular ways and to regard only the particular qualities of objects governs our relation to those objects (and to other subjects) and forms our truth claims. Dr. Phil Carspecken’s philosophy course, H510, reintroduced me to the Hegelian dialectic, where critical reflection inevitably transforms our awareness of the entanglement of our perceptions with the qualities and relations that originally derived from our concepts held about the world around us. Therefore, as education researches, in order to ground our truth claims in research studies, we operationalize our terms and explicate the methods employed so that all other researchers can perform the study.
Understanding the role of instructional technology leaders begins the overall research goal that seeks
to understand technology integration, its purpose, what it should it look like, and the consequence of replacing the “traditional” elements of instructional practice. The research goal is to begin with Diffusion of Innovations theory to examine the systemic and functional processes that culminate in the classroom setting. It will require embarking on research into more domain areas, including the instructional strategies employed with technology, the particular features of the instructional environment, and the support from instructional technology leadership. Building knowledge about these concepts would be valuable so long as the research conducted is systematic and iterative. It is clear that it will be a life-long research journey to answer the questions above:
However, there has been no consensus about what technology integration should look like (Sims &
Koszalka, 2008). In particular, there appears to be a misunderstanding about the role instruction plays when technology becomes a part a teacher’s pedagogical practice. Is technology integration similar to the adage about pornography: I know it when I see it? We expect school leaders to translate the potential for technology to improve student learning as evidenced from high stakes tests, but without explaining the factors required of teaching practice. Are test scores increasing because of the technology or despite it?
The theoretical foundation for my first-authored study developed from conversations with two IST
instructors. Dr. Thomas Brush introduced me to Everett Rogers’ Diffusion of Innovations, the process where change agents facilitate the adoption of innovation, which includes influencing the social system of an organization and framing the adoption as a desirable, direct and anticipated consequence (Rogers, 2003). In other words, change agents provide the foundation to propose that an innovation exists, that it is necessary, and that it will allow individuals to function within the organization better than they can now. Rogers used the phrase “relative advantage” or the degree that an innovation is perceived as better than the idea, habit, or thing it replaces. To overcome resistance to the innovation, change agents must understand the organizational culture and the roles individuals inhabit, including the variety of acceptance levels that individuals express when asked to transform their method of work. The elements to Diffusion of Innovations theory describes well the school leaders who provide the resources to support technology vis-à-vis the role they presume instructional technology leaders should possess.
Dr. Brush and Dr. Anne Leftwich helped me to explore the TPACK and SAMR models with regard to
technology integration. For instance, the TPACK framework views the intersection of three knowledge areas (content, pedagogical and technological) as equally weighted elements that inform instructional practice (Davies & West, 2014, p. 847). examining the elements of SAMR - Substitution, Augmentation, Modification, and Redefinition - becomes crucial when describing implementing instructional technology in K-12 classrooms if the technology is meant to support teachers AND allow teachers the time necessary (Frick, 1996) to discover and reflect on their pedagogical knowledge (PK) and content knowledge (CK) in order to realize the affordances in the form of technological knowledge (TK). The TK informs the PK and CK wherein the teacher realizes instructional goals that didn’t exist before they began teaching with a technology device.
A revised perspective of TPACK views the three knowledge areas as two levels, with pedagogy and
content at the very top, and technological knowledge at the very bottom. For example, Shattuck (2012) studied professional development programs where experienced teachers and new teachers mentored each other with technology integration. The experienced teachers had higher pedagogical content knowledge but lower technological knowledge than the new teachers. The teachers with higher pedagogical content knowledge were able to focus on key pedagogical needs. They were able to locate distinctive technology features that informed their decisions about which software application would provide the key affordances necessary to their instructional practice. The newer teachers focused greater attention to the broader possibilities of technology choices but were less able to make technology decisions that aligned clearly with pedagogical goals.
Although the original research purpose of my first-authored study was to revalidate the Brush and
Bannon (1998) survey, my participation in district leadership meetings and informal interviews with superintendents and building principals has revealed that the function of technology leadership is much more strongly related to “fixing things,” whereas instructional leadership is related to teaching the school curriculum. However, to understand instructional technology means examining whether the distinction between technology and instruction continues to be the dominant perspective as it relates to integrating technology in classrooms.
I’ve been involved in three research groups so far, all of which have helped me practice the skills and
knowledge acquired in the major and minor doctoral courses. Each group provided opportunities to engage in research design, construct clear research questions, develop practicable methods, and establish a consistent manner of data collection and analysis. In addition, my contribution has informed the development of my research practice and perspective.
Case-Based Reasoning (August 2013 to October 2015)
-
Dr. Krista Glazewski’s research group built on the research design processes introduced in R690. I had learned to develop research constructs from a nascent question about case-based reasoning, based on Dr. Janet Kolodner’s body of work. Through an intensive circular process between searching and reviewing the literature and discussion to refine our focus, our group learned strategies to saturate our knowledge about a research subject, which inform the quality of the proposed methods for data collection. Further, our group constructed survey questions, participated in the IRB review, conducted interviews, and analyzed data, and shared equally in the writing and revising of the study content. Our research deliverables culminated in a poster session at the 2014 AECT conference and a completed research manuscript for publication.
Problem-Based Learning, secondary school (January 2015 to present)
-
Dr. Thomas Brush and Dr. Kyung-bin Kwon’s research group has allowed the opportunity to engage more in data analysis of recorded observations. Although other longer-term members had visited the research site and collected video of secondary student activities, the members of our group have met often on several occasions to review recordings and code behavior. In addition, our group meets regularly in between coding sessions to evaluate our methods and coding progress in order to achieve inter-rater reliability. The analysis has provided much practice to achieve consistency not only in my own methods but also in relation to the activities of colleagues.
Problem-Based Learning, post-secondary (October 2015 to present)
-
Dr. Glazewski and Dr. Cindy Hmelo-Silver’s research group involves greater emphasis on evaluating recorded interview and observations. I had the opportunity summarize a collection of recorded interviews from a group of students attending the University of Texas at El Paso and present the summary to our research group. Also, I recently have been invited to meet regularly with Drs. Glazewski and Hmelo-Silver to review instructor interviews and explicate a more nuanced analysis of PBL vis-à-vis the higher education context. The experience has allowed a clearer appreciation of the demands and expectations that influence the secondary and post-secondary research settings.
-
Ability to review and synthesize literature
-
Ability to write research proposal and design and conduct research studies
-
Develop scholarly writing and presentation skills
-
Knowledge of theories related to IST
-
Knowledge of appropriate methods for conducting qualitative research
-
Apply research methods in appropriate context
-
Ability to review and synthesize literature for potential research
Completed: The literature review What Do We know About the Effects on Student Learning When Mobile Touchscreen Devices Have Been Implemented in K-12 Classrooms? was an attempt to synthesize the current state of knowledge about mobile devices in K-12 classrooms. The gaps in the literature address not only barriers to technology but also addresses technology, specifically mobile devices, as a potential barrier to learning.
Completed: The literature review Characteristics of K-12 Technology Leaders: Their Role in Technology Integration synthesized the primary roles of K-12 technology leaders, including the expectations and skills to support technology integration I the classroom. The review included gaps in the literature and implications for further research.
-
Ability to write research proposal and design and conduct a research study
Completed: I have written a research proposal, developed a survey instrument, applied for and received IRB approval, and begun data collection. Analysis of the results should be completed in early spring 2016. The goal is to conclude analysis of the survey results finalize the research study for publication: Is There Room for K-12 Instructional Technology Leaders? A Survey of Public School Superintendents in Indiana.
-
Develop scholarly writing and presentation skills
Completed: In Fall 2013, I completed an article critique on Schools going mobile: A study of the adoption of mobile handheld technologies in Western Australian independent schools for Dr. Glawseski's R690 course.
Completed: Participated in a poster session for the Case-based Reasoning research study for the 2014 AECT Conference.
Completed: Spring 2015 IST Conference. I was the primary presenter at a round table discussion introducing the research proposal of Characteristics of Technology Leaders
Completed: Spring 2015 Indiana CTO Conference, Hoosier Room. I was the primary presenter at K-12 school technology leader session
Completed: Fall 2015 AECT Conference. I was the primary presenter at a concurrent session on Leadership Development from K-12 to Higher Education.
-
Knowledge of theories related to IST
Completed: I’ve completed foundational coursework in IST, including knowledge of core theories and their application to my participation in three IST research groups.
IST courses and minor courses completed to gain understanding of the IST field and scholarly knowledge.
- R711: Readings in Instructional Technology (Fall, 2013)
- R690: Application of Research Methods to IST Issues (Fall, 2013)
- R521: Instructional Design and Development (Fall, 2013)
- R541: Instructional Development and Production (Spring, 2014)
- Y502: Intermediate Statistics Applied to Education (Spring, 2014)
- Y521: Methodological Approaches to Educational Inquiry (Spring, 2014)
- Y611: Qualitative Inquiry in Education (Spring, 2015)
- R561: Evaluation and Change (Sum. 2015)
- H510 Foundations in Educational Inquiry (Fall 2015)
The following courses will provide additional understanding of knowledge about IST theories.
- Y525: Survey Research Methodology (Spring, 2016)
- R621: Needs analysis and assessment (Fall, 2016)
-
Knowledge of appropriate methods for conducting qualitative research
Completed: I completed Dr. Lucinda Carspecken’s Y611 Qualitative Inquiry in Education and Dr. Phil Carpsecken’s H510 Foundations in Educational Inquiry. Both courses deepened my foundational understanding of qualitative methods and the historical philosophies that provided the guiding framework for inquiry. In spring 2015, I conducted two qualitative data collection tasks.
Y611 Observation, Y611 Interview
-
Apply research methods in appropriate contexts
Completed: I have collected data that has applied both qualitative and quantitative methods, including discussion and practice in research groups, coursework, and in my first-authored study, including providing descriptive statistics and conducting interviews and observations.
Survey Statistics, Survey Questions, Interview Protocol
References
Davies, R. S. & West, R. E. (2014). Technology integration in schools. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of Research on Educational Communications and Technology (pp. 841-850). New York, NY: Springer.
Frick, T. W. (2015, April 24). Stop punishing teachers: What we should be doing instead to improve education. Retrieved from Educology, Department of Instructional Systems Technology, School of Education http://educology.indiana.edu/Frick/StopPunishingTeachers.pdf
Guba, E. G. and Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions, and emerging confluences. In: N.K. Denzin and Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed.). Thousand Oaks, CA: Sage.
Rogers, E.M. (2003). Diffusion of Innovations. New York, NY: Free Press.
Shadish, W. R., Cook, T. D & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal Inference. Construct Validity and External Validity (pp. 83-87). Boston , MA: Houghton Mifflin Company.
Shattuck, G. (2010). Understanding school leaders’ role in teachers’ adoption of technology integration classroom practices. In M. Orey, S. A. Jones, & R. M. Branch (Eds.), Educational Media and Technology Yearbook 35 (pp. 7-28). New York, NY: Springer. doi: 10.1007/978-1-4419-1516-0_2
Sims, R. C. and Koszalka, T. A. (2008). Learning with technology: Using computers as cognitive tools. In J. M. Spector, M. D. Merrill, J. van Merrienboer, & M. P Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 569-575). New York: Lawrence Erlbaum Associates/Taylor & Francis Group.
