Sign In, Renew, Sign Up

Search AIA

Search AIA Go

Practicing ArchitecturePracticing Architecture

Page Tools

CMD Insight for Architects

Advertisements

Description: Stellent_Header

AEC Research Review: A Knowledge Pyramid Model for Determining Trustworthiness

By Richard L. Hayes, AIA, PhD
Director, Knowledge Resources, the American Institute of Architects

The AEC Research Pyramid was developed during the 2012 and 2013 AIA Research Summits, with the premise that the knowledge strata for the academy and practice are connected, but not identical. There are multiple varieties of research within the architectural, engineering and construction (AEC) fields. Individuals and organizations run the full gambit including independent scholars, AEC firms, business consultants and academic institutions. A key to appreciating the research undertaken and reported upon depends on the devotion of the scientific method an individual or organization adopts. Within the AEC sector both qualitative and quantitative research can be done in a way that is consistent with the scientific method. Differences in approach are to be expected, the key is that the application of the scientific method helps to ensure research is conducted in an unbiased way. The research conducted in practice and academy settings depend on different educational contexts, as well as professional experiences. As part of the BRIK initiative, the results from the Research Summits’ AEC Research Pyramid are to be posted in BRIK and expected to assist in clarifying research submissions. (Figure 1. Knowledge Pyramid Model)

Overview

Shared knowledge on the academy side is based on rigor. For practice, knowledge is based on applicability. The academy is driven by peer review supporting a tenure track career. Practice is based on usefulness of knowledge for the direct application to service and project delivery. For the profession to have the full advantage of research from both spheres there must be recognition of the value of the overlay and connection of these two bodies of knowledge. It should be noted that as a piece of research “moves up” the pyramid, it does not necessarily increase in applicability, only that the methodology followed more closely resembles the scientific method.

An apparent gap of credibility between the two seems to lie on what was termed “industry review.” On the practice knowledge pyramid, research does not qualify at the level of rigor expected in academic settings. This “industry review” is a product of architects publicly sharing their knowledge within the AEC industry for comment. Research findings in this setting can be relied upon, but practitioners may choose to proceed on the basis of relatively speculative findings.

Twin concepts of reliability and validity must be considered to determine trustworthiness. Reliability is the degree to which an evaluation process yields steady and dependable outcomes. Validity is the quality of being empirically sound. So the underlying unifier for the two sides of the pyramid above is a third side that has to do with achieving trustworthiness, i.e. can you trust the study? This aspect has four levels: D – not reviewed, C – reviewed for structure (grammar, spelling), but opinion not reviewed, B –study opinion reviewed, A – study reviewed for trustworthiness. (Figure 2. Level of Review)

CARS

As outlined above, trustworthiness implies some degree of a peer review process. Trustworthiness suggests a peer review process that takes into account CARS. CARS is an evaluation of a source (research) using Credibility, Accuracy, Reasonableness, and Support. The characteristics of these attributes are below.

Credibility

          Author and credentials listed

          Well edited in terms of grammar, spelling

          Positive well balanced tone

          Relevance

Accuracy

          Date

          Recently published or considered seminal

          Succinct

          Original

Reasonableness

          Tone or language that implies unbiased attitude

          No conflict of interest

          Specific points of fact

          Reliable

Support

          Source for data or statistics provided or referenced

          Documentation provided or referenced

          Corroborating sources listed

          Well-balanced point of view

Overview of the research pyramid development

The knowledge pyramid model for determining trustworthiness had its genesis during the 2012 AIA Research Summit. During that summit there was discussion on metrics and measurement of AEC research. Out of those discussions the pyramid emerged. The following year’s summit revisited the model with discussions and revisions. Since the conclusion of the 2013 summit, further revisions have occurred. What follows is a brief overview of the summits’ discussion with regard to the pyramid. (Figure 3. From 2012 AIA Research Summit, Pyramid’s first mention)

Metrics/Measurements Discussion

The major outcomes for this session were items to take into account when establishing metrics/measurements:

  • Applicability (can it be used?)
  • Replicability (can the experiment be run again and achieve the same results?)
  • Transparency (is all the data available for review?)
  • Findings and Supporting Data (outcome measurements in a useable manner)
  • Usability (was it used by others?)
  • Rigor (degree of precision coupled with supporting documentation)
  • Quality (standardized operations and high degree of integrity)
  • Cost Impact (return on investment)

Summit participants looked at several options to rank and rate research via a simple matrix diagram. The idea was to establish an easily understood graphic that would help in establishing metrics and measurements.   Also the graphic concept was to aid in depicting differences between academy tenure and professional evaluation of research documents. (Figure 4. From 2012 AIA Research Summit, Pyramid Refined)

The idea is to have a regular system where by a piece of research can evaluated by parameters of:

  • Level of Review
  • Degree of Supporting Documents
  • Degree of Simulation

The 2013 Summit revisited the pyramid as a benchmarking tool. A benchmarking model that holds potential for architectural research can be called a “pyramid of inquiry” (POI). The pyramid holds promise, because it combines the matrix concept and the sliding scale. A three sided POI pyramid can represent three realms of research: practice-based, academic-based, and review of rigor. The multiple faces could also be designated for specific undertakings, such as design research, educational research, and forensic studies. In each case the reflection of rigor exists by the tip of the pyramid representing the highest degree of rigor, and statistical validity is assumed. By varying the focus of the pyramid faces, the model becomes scalable and can be applied to the design world.

A refined pyramid model was included in the 2013 Summit Report. By having the pyramid context defined, this enables dialogue allowing the AIA to learn and document value propositions for firms doing research, i.e. what constitutes research for the practitioner? How do researchers help the general practice of architecture?

Description: white-20x10

Figure 1. Knowledge Pyramid Model

Description: enlarge-icon

Figure 2. Level of Review

Description: enlarge-icon

Figure 3. From 2012 AIA Research Summit, Pyramid’s first mention

Description: enlarge-icon

Figure 4. From 2012 AIA Research Summit, Pyramid Refined

Description: enlarge-icon

 

Read glossary and annotated bibliography

   

Recent Related:

AIA and NIBS Launch Free Building Research Portal

 

Back to AIArchitect February 21, 2014

Go to the current issue of AIArchitect

 

Footer Navigation

Copyright & Privacy

  • © The American Institute of Architects
  • Privacy