Supplemental Reports

Denis Gray, Lou Tornatzky, Lindsey McGowen
Completed 2012

The Industry/University Cooperative Research Center (IUCRC) program was designed to enhance the US research, education and technology transfer infrastructure by promoting stronger technical and organizational ties between university and industry. Evaluation efforts undertaken during the IUCRCs thirty-plus years of operation have provided persuasive and methodologically robust evidence that it has been achieving these objectives. However, like other federal S&T programs, little has been known about the extent to which the IUCRC program achieved another important program objective . the establishment of long-term partnerships that were self-sustaining.

Based on a companion study (McGowen, 2010), we were able to demonstrate that that roughly two-thirds of all the IUCRCs that were launched over the past thirty years were still operating. By almost any metric, this is an enviable record of success. Importantly, although these Centers are no longer supported by NSF they continue to have a large indirect effect on achieving program objectives including leveraging of government funding, student training, scientific achievement, technology transfer and commercialization. However, this primarily quantitative study, failed to shed much light on the factors that contributed to the long-term survival or failure of these Centers. Given this background, the overarching objective of the current study was to gain a better understanding of why and how some Centers achieve sustainability while others do not.

Building on archival data collected as part of the ongoing IUCRC evaluation effort and new interviews with key informants involved in the transition from a government-funded program to self-sustainability, we developed a collection of case studies that attempted to highlight both the factors that influenced survival and the different post-NSF paths to Center sustainability that different Centers could take.

An ad hoc multiple case analysis of four IUCRCs that failed either early or later in their program history (Chapter 2) highlighted several factors that contributed to a Center ¡°unraveling¡± including: absence of key I/UCRC ingredients (e.g., doctoral level programs); poor or no transition planning; withdrawal of institutional support; problems of leadership aggravated by poor succession planning; and cascading effects of several of these factors.

Four case studies highlight how, with effective leadership, Centers can not only navigate these and other challenges to sustainability but in some cases reinvent the IUCRC model so it was better suited to unique local needs and circumstances. Chapter 3, Reinventing the IUCRC Model, tells the story of the Center for University of Massachusetts/Industry Research on Polymers (CUMIRP) and how it developed a hybrid center model. Chapter 4, Success Through Fidelity to the I/UCRC Model, tells the story of how the Advanced Steel Processing & Products Research Center (ASPPRC) at Colorado School of Mines used the core I/UCRC model to meet the challenges of an industrial sector that was both globalizing and shrinking. Chapter 5, Transformation of a Small University IUCRC, tells the story of how the Center for Advanced Communication at Villanova University (CAC) reinvigorated a declining Center and helped build a strong graduate program by moving to a more contract research mode of operation. Finally, Chapter 6, IUCRC as Capacity Building Strategy for State-based Economic Development, tells the story of Ohio State.s Center for Welding Research and how very early in its development it morphed into the not-for-profit Edison Welding Institute (EWI), one of the world.s largest and most respected manufacturing-focused research institutes.

Taken together we hope these cases and the lessons that they convey will help both policy makers and local center leaders navigate a successful path to partnership-based program sustainability.

——————————-

Lindsey McGowen
Project Completed 2012

Industry/University Cooperative Research Centers (I/UCRCs) are supported by funding from the National Science Foundation but, like other center programs, are expected to achieve self-sufficiency after a fixed term (ten years). However, there is little evidence about the extent to which government funded programs are able to make this transition. McGowen (2010) was able to document the level of sustainability achieved by formerly funded I/UCRCs, and document several structural and environmental variables that predict sustainability outcomes. The current study builds on that research by taking a longitudinal approach to examining changes in program sustainability over time, and investigating the impact of measures of stakeholder support and leadership transitions in addition to the environmental, organizational, program, and individual domain variables previously identified. Archival and survey data were used to explore program sustainability of IUCRCs post-graduation from initial grant support.

Results indicate that the majority of formerly funded IUCRCs are still operating, despite a small decrease in sustainability over time. When comparing sustained Centers to actively funded I/UCRCs, the sustained Centers appear to have less adherence to IUCRC model activities but they have been able replace the resources and structural capacity associated with the end of their IUCRC grants and are actually achieving greater outcomes than are actively funded Centers; an indication that only the strong survive.

Predictive analyses were able to identify environmental, organizational, program, and individual domain variables that were associated with Center status and change in status over time. Centers who receive enough support from NSF to become well-established, do not conclude their NSF support during economic downturns, have committed long-term directors, and produce valuable human capital benefits for their members are more likely to continue to operate over the long term. In terms of predicting what variables may account for the level Centers’ continued activities, structural capacity, and benefits to stakeholders, these outcomes tend to be higher for Centers with high research productivity (as measured by their rate of publishing), that have a significant positive impact on their members, provide members access to valuable human capital in the form of Center trained graduate students hired by members, and whose directors are not distracted by non-Center administration and other tasks.

This study was the first to longitudinally examine program sustainability for cooperative research centers. Results can be used to inform the policy makers and cooperative research center stakeholders about the transition process to program sustainability. While these results are informative, additional research is needed to understand cooperative research center sustainability and to develop a more robust theoretical model of program sustainability in general.

——————————-

Lindsey McGowen
Project Completed 2010

Industry/University Cooperative Research Centers (I/UCRCs) are supported by funding from NSF but, like other center programs, are expected to achieve self-sustainability after a fixed term (ten years). However, there is little evidence about the extent to which government funded programs are able to make this transition. This study attempted to identify the factors that predict Center sustainability after they have graduated from NSF funding. Archival data and qualitative interviews with Center Directors were used to explore program sustainability of I/UCRCs post graduation from initial grant support. The study examined environmental, organizational, program, and individual level constructs to predict Center status, fidelity to the IUCRC program model, and sustainability in terms of continued infrastructure, program activities, and outcomes. Results indicated that three quarters of formerly funded IUCRCs are sustained beyond the end of their grants and that they maintain a high level of structural fidelity while making modifications to Center assessment processes. Sustained Centers also exhibited a comparable level of continued program structures and outcomes when compared to actively funded IUCRCs. Predictive analyses indicated that IUCRC program sustainability is impacted by environmental, organizational, and program level factors. The results inform the transition process for Centers currently funded under the IUCRC program as well add to the body of knowledge on program sustainability developed in other content areas.

Denis Gray & Drew Rivers
Project Completed 2012

A limited body of research has begun to examine the impact of the Industry/University Cooperative Research Centers (I/UCRC) on the creation and enhancement of scientific and technological human capital. However, the impact of the program on a critical participant group, center directors, had yet to be investigated. The overarching goal of this study was to determine the professional trajectory and achievements of I/UCRC directors and the extent to which these outcomes can be attributed to their IUCRC experience and training. Building on the emergent literature on the effects of center participation on faculty, we applied mixed methods of data collection including a focus group, web survey, curriculum vitae (CV) analysis, and telephone interviews. We found the experience of serving as an IUCRC director can have significant and positive impacts on the human and social capital of faculty members. However, some directors reported excessive workload and expectations given the funding provided by NSF. Further, as a director, the faculty member can enable the careers and outcomes of individuals and organizations involved with the center. Through regression analysis we explore factors that predict the extent to which the IUCRC program influences career outcomes and overall work satisfaction. We examine different career opportunities in both academic and industry leadership positions reported by former and current directors, and summarize the career trajectories of IUCRC directors along three lines: administrative, technical, and technical leadership. Finally, we summarize former and current IUCRC directors’ recommendations for improving the IUCRC program by using a director’s lifecycle model. In our conclusions, we highlight the spillover benefits of IUCRCs to their host universities, offer a marketing piece that highlights these benefits, highlight funding changes that might mitigate director complaints about excessive workload and expectations and propose a simple method of screening both faculty members and their universities for their IUCRC readiness.

Denis Gray & Drew Rivers, with George Vermont
Project Completed 2012

This report summarizes findings from a study designed to strengthen the impact evaluation efforts of the NSF Industry/University Cooperative Research Centers (IUCRC) program. With modest annual financial support from NSF in combination with membership fees from industry, IUCRCs perform industrially-relevant research that is carried out primarily by faculty and graduate students. The NSF IUCRCes current evaluation strategy calls for the collection of data primarily through the assistance of on-site evaluators and episodic documentation of \breakthroughs.. The study had three objectives: 1.) To assess the strengths and weaknesses of the current impact assessment strategy; 2.) To assess the feasibility of improving the programfs ability to obtain credible and persuasive quantitative estimates of economic impact; and 3.) To make recommendations based on these findings for a strategy for routinizing the collection of such impact data on an ongoing basis.
Based on our analysis of impact data collected, we found that the current evaluation approach appears to do a good job of addressing the programes explicit partnership and capacity building objectives. Data currently collected documents the leveraging effect of IUCRC financial support, the establishment of valuable partnerships, immediately realized R&D impacts and descriptions of technology \breakthroughs.. Unfortunately, most informants who had benefited were unwilling or unable to provide economic impact estimates for R&D and commercialization outcomes. When estimates were provided they were often given as forecasts of future impacts.

In order to try and remedy these shortcomings, we examined the value of an alternative assessment strategy: rather than interview all possible beneficiaries, we conducted interviews with firms that had been nominated as potential \high impact beneficiaries. at mature IUCRCs and we provided those informants with confidentiality to protect sensitive information. Although no assessment effort will be able to quantify the economic impact of all the important impacts of the IUCRCs. This new assessment strategy proved to be very successful in obtaining credible quantitative estimates of the economic impact of IUCRCs. We approached three mature IUCRCs and found informants were able and willing to provide estimates of the economic value of a number of specific impacts, including improved R&D efficiencies; improved or new processes; improved or new products; and spillover benefits to technology adopters. Aggregating results across the three centers we found a total present value of realized impacts of $1.28B, generated from NSF program investments of approximately $18.5M in present value in these three centers. Each dollar invested by NSF helped to create $69.4 in benefits, with a net present value of $1.27B. Since many of the technologies referenced by the informants were in the earliest stages of being deployed and/or sold, informants also provided us with forecasted economic impacts for the next two to five years which were sometimes multiples of what had already been realized.

Our analyses strongly indicate that the IUCRC program is having a significant and measurable economic impact on member companies and other external beneficiaries. Since we picked centers that were judged to be \successful,. we do not know how representative these findings are. However, if one takes a portfolio approach to evaluating the IUCRC program at a national level, these findings alone, based on a very small percentage of the currently funded centers and participating members, appear to provide sufficient justification for NSFes investment in the overall IUCRC program over many years.

Although there are a few methodological challenges to overcome before the approach we used in this assessment can be employed on a routine basis to collect data from IUCRCs, we believe the program n-site evaluators are in a very favorable position to continue this type of assessment on an ongoing basis. Several recommendations are made to try to achieve this objective.

In 2007 the National Science Foundation’s Industrial Innovation and Partnerships (IIP) division initiated the Supplemental Opportunity for SBIR/STTR Memberships in Industry/University Cooperative Research Centers (I/UCRCs). This supplement opportunity aimed to accelerate the innovation process by “partnering industry-relevant academic research with commercialization focused small business research.” In brief, the supplement involved providing Phase II NSF SBIR/STTR firms with a subsidy that would allow them to join an I/UCRC of their choice for up to two years (the subsidy covered about 90% of the cost). Over a five-year period, we estimate that about 14% of the SBIR/STTR firms that were eligible for this opportunity took advantage of it (N=72), about one-third of centers had such a member and NSF provided a total of about $4.4 million to support this “experiment”.

This report is the first systematic evaluation of this innovative cross-program supplement. In our assessment, we examine the supplement through two lenses: a need-based rationale that might lead to an incremental improvement of both programs, and a “combinatorial innovation” framework whereby SBIR/STTR membership in an I/UCRC provides for a mutual exchange of resources and services. This framework has the potential to produce synergistic interactions between the programs that yield more far reaching impacts. We address the question of impact with a variety of data sources and methods, including secondary firm-level and center-level data, surveys with I/UCRC directors, and in-depth, structured interviews with SBIR/STTR firm representatives.

We find that the supplement had a modest impact on the I/UCRC membership profile, with SBIR/STTR firms adding to the frequency of small businesses engaged with I/UCRCs by about 10%. However, the firms that are added via the supplement are statistically significantly smaller, younger and include a more diverse ownership than I/UCRCs have reported in the past. In short, the supplement increased participation in I/UCRCs by firms that appear to be micro-enterprises or start-ups.

Among I/UCRC directors we find a favorable benefits-to-costs ratio in having these small firms involved in center operations, center research and education activities. While directors most frequently mentioned the additional membership support they received due to these memberships, they also highlighted the active involvement of these firms in student and project mentoring and in discussions among the center’s Industrial Advisory Board (IAB) as benefits. The high turnover rate of these members was cited as a cost by a large number of directors. Nonetheless, the vast majority of directors reported they were likely to recruit SBIR/STTR members in the future.

At the same time, we find that most SBIR/STTR firms reported multiple benefits and very favorable outcomes related to I/UCRC participation. Benefits range from improved R&D efficiencies, enhanced social and human capital, and accelerated commercialization. In some cases there appears to be a synergistic effect between these benefits for participating firms. From a human capital perspective, the vast majority of SBIR/STTR firms reported making valuable connections to faculty and students and the majority reported making similar connections to other center members. Importantly, a very large percentage of firms reported continuing these very productive interactions on an informal basis after their membership ended. The majority of firms also reported R&D benefits like saving money or time on their internal research and project development efforts. Finally, the majority of SBIR/STTRs reported a number of commercialization-related benefits including identification of new applications for the technology they were developing, improvements in the products and services, and IP related impacts. About 20-30% of firms reported concrete commercialization impacts like attracting new investors, adding jobs, and introducing new products or services. Some firms mentioned unproductive aspects of the supplement, most related to specific operational features of their involvement.

Although most firms choose to not maintain their membership after the supplement ends, almost 90% of them report their involvement in the supplement was worth both the time and money they invested in the activity. We incorporate short case summaries to highlight the rich interactions between SBIR/STTR firms and I/UCRCs and the synergy between the human capital, R&D and commercialization-related outcomes generated by those interactions. The report concludes with recommendations for both IIP and NSF that are based on our findings and feedback provided by directors and firms.

The study will empirically evaluate the effectiveness of two NCSU entrepreneurial education programs (undergraduate and graduate) and identify the individual and contextual factors associated with entrepreneurial propensities and effectiveness. Little empirical research exists evaluating the outcomes of entrepreneurial education programs; what does exist is of mixed quality with few comparison groups.

The study’s operating hypothesis will be that entrepreneurial education outcomes will be higher for the NCSU program alumni than for a group of similar non-alums, and these direct effects are mediated by a number of individual (personality and cognitive/skill set) and contextual variables. For example, a number of studies have found that self-efficacy and over-optimism/ non-counterfactual thinking are personality characteristics common to entrepreneurs. Cognitive abilities such as opportunity recognition, creativity, and risk assessment, and skill sets in business and resourcefulness/networking will be evaluated as will contextual factors such as familial, cultural, community, and workplace/ organizational variables such as roles, organizational culture, leadership, etc.

The study will employ a quasi experimental design employing a matched control group selected via stratified random sampling (matching will address the criterion group’s self-section threat to internal validity). Participants will be matched on SAT test scores (which are correlated with post-college career success) and parent/family entrepreneurship (which is correlated with entrepreneurial tendencies).

Two kinds of data will be collected (individual/contextual characteristics and entrepreneurial behaviors/outcomes) from the respondents via a web questionnaire and other data gathering devices. Targeted individuals will be contacted by mail, telephone, and email to increase response rates. Factor analysis (both exploratory and confirmatory) will be conducted to fine-tune our model and validate our measures, and the three dependent variables (entrepreneurial outcomes of proclivity, activity, and success) will be assessed for the two groups of program alumni and the two matched control groups. Hypothesis testing and multiple regression/multivariate analysis will be conducted and a predictive model will be developed and tested utilizing structural equation analysis where appropriate.

Results of the study will include the development of a new scale of entrepreneurial outcomes. In addition the project will produce a causal model of key factors for entrepreneurial success that will, by informing educators, lead to the development of even more effective entrepreneurial training/education programs that produce more successful entrepreneurial outcomes.

Craig Scott
Project Completed 2010

The National Science Foundation is constantly striving to more fully document the impact that I/UCRCs have on the development of technological breakthroughs, technology transfer and commercialization. Previous compendia have been used by the I/UCRC program to demonstrate the scope of high quality, industrially relevant research outcomes produced by the program. The proposed project will gather, organize, catalogue and illustrate noteworthy program-related technological breakthroughs that have resulted from I/UCRC research and subsequent technology transfer activities.

The deliverables from the proposed project would make it easier for policy makers and researchers to access breakthrough descriptions from all I/UCRC scientific areas. Deliverables will also be useful as marketing tools for informing industry and the scientific community of the benefits of such programs, and could pave the way for studies of economic impacts of cooperative research.

https://www.nsf.gov/awardsearch/showAward?AWD_ID=0902804&HistoricalAwards=false

Evaluator exit interviews of industrial sponsors opting not to renew support have been a required part of the I/UCRC evaluation protocol for over 20 years. Exit interviews were developed to record sponsor recommendations for center improvements as well as the sponsors’ reasons for leaving, including their expectations, surprises, and disappointments. However, there has never been a centralized collection mechanism for this interview data nor a formal analysis of interview results across the I/UCRC program.

This project delivered the protocols and mechanisms necessary to collect exit interview data across all centers and, once one year of data was compiled, the analysis focused on the value of the exit interview process to key stakeholders.

Since the early 1980s the US government through federal legislation has worked to increase public-private partnerships to help drive industrial innovation and economic progress (National Science Board, 2006). Cooperative Research Centers (CRCs) are one such mechanism, enabling industrial organizations to collaborate with both universities and government agencies. These university-based CRCs operate as linkage mechanisms (Gray, 1998), bridging the culture gap between academia and the private sector. Understanding the motivations and processes through which these partnerships initiate and evolve is important to their continued success. From existing research we know what environmental and organizational factors signal an increased likelihood for partnerships to occur. However, beyond these initializing conditions little is known about how organizations discover potential partners and subsequently decide whether to pursue a formal partnership arrangement. This current study applied a mixed methods approach to identify factors within organizations that could explain how industry-university partnerships happen. Two preliminary studies were conducted to explore pre-collaborative exchanges between university-based CRCs and their prospective member organizations. These first two stages of research revealed underlying communities of university researchers, industrial technologists, and government scientists. Within these communities reside networks of actors engaged in dynamic relationship exchanges that propagate formal partnership considerations. Further, semi-structured interviews with organization representatives brought to light a varied and often increasingly elaborate process regarding decisions to partner with university-based CRCs. The final stage of research administered a structured survey to a sample of industrial and public organizations. The decision process is described as it unfolds within organizations considering CRC membership. Further, a series of regression models identified the unique and relative effects of decision outcome predictors across several domains of analysis. I found support for network-based perspectives on the development of industry-university partnerships. However, the influence of network relationships rested primarily on the initiation of the partnering decision. Technical and non-technical characteristics of the CRC, as well as sub-organizational and individual variables, were found to be most predictive of actual decision outcomes. Implications of this research for CRC directors, prospective member organizations, and policymakers are offered.

Funding: CERSP and support from NSF-IUCRC Evaluation project.

Andrea Lloyd
Project Completed 2007

The creation of collaborative partnerships between historically black colleges and universities (HBCUs) and predominantly white institutions (PWIs) is regarded as a very promising mechanism for strengthening the educational pipeline for student populations that are underrepresented in research fields of study and for strengthening the research infrastructure of HBCUs. In spite of the growing popularity of HBCU-PWI research partnerships, there is virtually no data on the characteristics of these partnerships and the factors that affect partnership outcomes. The current research sampled a population of HBCU-PWI partnership principle investigators (HBCU PIs, n=8; PWI PIs, n=8). For the purposes of comparative analyses, a sample of PWI-PWI partnership PIs (n=10) also was obtained. Partnership PIs responded to forced-choice and open-ended items during a structured interview. Based on their responses, the major characteristics of these partnerships are described, and comparisons are made between the PI groups. Regression analyses were performed to determine which factors predict partnership outcomes, including satisfaction with the partnership process and project outcomes, perceptions of partnership success, and PI willingness to participate in similar partnerships in the future. Qualitative analyses of responses to open-ended interview items were also conducted. Study results indicate that PIs of HBCU-PWI partnerships regard their partnerships as successful and are satisfied with project outcomes and partnership processes. Furthermore, the data show that HBCU-PWI partnerships have goals that are more focused on the research training underrepresented groups than do PWI-PWI partnerships. The extent to which university resources were allotted for PIs’ participation in their partnerships, PI perceptions of partner capability, the location of face-to-face meetings, and the number of partners involved were found to predict some partnership outcomes. Implications for partnership strategy and public policy are also discussed.

Jennifer Schneider
Project Completed 2007

Graduate students who participate in Cooperative Research Centers are perceived as having educational advantages. Cooperative Research Center alumni were rated superior in job performance when hypothetically compared to their organizations’ peers by themselves and their supervisors (Fitzsimmons, Grad, & Lal, 1996; Parker, 1997; Scott, Schaad, & Brock, 1991). Center alumni were also rated as being more prepared and needing less initial job training when hypothetically compared to their organizations’ peers by their supervisors (Ailes, Roessner, & Feller, 1997; Fitzsimmons et al., 1996). Exposure to multiple disciplines and an emphasis on teamwork were the areas that center alumni reported as having the most positive impact on their career ( Fitzsimmons et al., 1996). Despite these positive findings, this literature includes a number of shortcomings including limited statistical analyses, use of retrospective and/or hypothetical assessments and a failure to examine empirically which center mechanisms and experiences account for these differences.
A cross-sectional predictive analysis was conducted to identify which individual center mechanisms positively or negatively influence graduate student outcomes. Data was collected from graduate students (n=190, 37% useable response rate) working in National Science Foundation’s I/UCRC and STC programs (34 centers, 81% response rate) via a web-based questionnaire. Predictive variables include individual characteristics, center characteristics, interactions with center affiliates, project involvement, thesis/dissertation research, formal center training mechanism (activities the center offers), and center experiences. Student outcomes include satisfaction, perceived benefits (non-technical and technical skills), organizational commitment, scholarly achievements (publications, patents, publications), career goals (desire to work in industry, government, or academia), and feelings of a competitive advantage.
Preliminary results indicate that key predictive variables include: Multidisciplinary Center Experience; Experiential Expanded Center Experiences; Project Activity and Involvement; frequency of interactions with Center industry members.