SciSIP AWARDS 2011 & 2012

Adoption & diffusion of knowledge:

1. Tracing Influence & Predicting Impact in Science (James Evans and Andrey Rzhetsky, University of Chicago)

2. The Long-Term Regional Economic Impacts from Public Investment in University Research (Shawn Kantor and Alex Whalley, National Bureau of Economic Research)

3. Organizations and the Diffusion of Scientific Knowledge (Scott Stern and Michael Bikard, Massachusetts Institute of Technology)

4. Incubators of Knowledge: Predicting Protégé Productivity and Impact on the Social Sciences (Cassidy Sugimoto, Ying Ding and Stasa Milojevic, Indiana University)

5. The NIH Public Access Policy: Potential Impact on Physicians and Community Health Organizations (John Willinsky, Stanford University)

Understanding the impact of structures/process on science:

6. The Evolving Research Enterprise (Maryann Feldman and Michael Roach, University of North Carolina at Chapel Hill)

7. Managing Community: The Organization and Management of Federal Research Funding Agencies (Michael Piore, Massachusetts Institute of Technology)

8. What Model for Public-Private Partnerships? Lessons from Existing Consortia for Administration of the U.S. National Network for Manufacturing Innovation (Erica Fuchs and David Hounshell, Carnegie Mellon University)

9. A Comparative Study of Structural Influences on User-Engaged Ecology Research (Mark Neff, Allegheny College)

10. ICES-GMU Workshop on Internationalization & Competitiveness (Eskil Ullberg and Daniel Houser, George Mason University)

11. Empirical Studies of Innovation in Health Care Markets (Heidi Williams, National Bureau of Economic Research)

12. EAGER-ORCID: Investigating ORCID as an accelerator of science of science policy (Ian Foster, University of Chicago)

Advancing understanding of entrepreneurship & innovation:

13. Atlanta Competitive Advantage Conference PhD Student Workshop (William Bogner, Georgia State University)

14. Inter-Industry Differences in the Antecedents and Consequences of Industrial Scientists Mobility and Entrepreneurship Decisions ( Rajshree Agarwall-Tronetti and Seth Carnahan, University of Maryland College Park; Martin Ganco, University of Minnesota Twin Cities; and Benjamin Campbell, Ohio State University)

15. NSF Patent Data Workshop (Alexander Oettl, Georgia Tech Research Corporation)

New approaches to studying science & innovation:

16. Understanding Innovative Science: The Case of the Wisconsin Institutes for Discovery (Daniel Kleinman, Gregory Downey and Noah Feinstein, University of Wisconsin Madison)

17. Evaluating the Effect of Cyberinfrastructure on Universities' Production Process (Amy Apon, Linh Ngo and Paul Wilson, Clemson University)

18. Sensible Science: A Sociometric Approach to Collaboration in Synthesis Groups (Edward Hackett, Arizona State University)

19. Incentives for Researcher Profile Maintenance and Access, and Their Value in Science and Innovation (Erik Lium, Claire Brindis, Mini Kahlon and Tuhin Sinha, University of California San Francisco; and Ian Foster, University of Chicago)

20. Scientific Collaboration in Time (Carl Lagoze, University of Michigan Ann Arbor and Steven Jackson, Cornell University)

Implementing science policy:

21. Science of Science and Innovation Policy: Principal Investigator Conference, 2007-2011 Awards (Kaye Husbands Fealing, National Academy of Sciences)

22. Balancing the Portfolio: Efficiency and Productivity of Federal Biomedical R&D Funding (Margaret Blume-Kohout, University of New Mexico and David Newman, University of California Irvine)

23. Effects of Immigrant Scientists and IPRS on Innovation (Petra Moser, National Bureau of Economic Research)

24. Estimating the Economic and Scientific Impact of Federal R&D Spending by Universities (Jason Owen- Smith and Margaret Levenstein, University of Michigan Ann Arbor)

25. 2012 Science and Technology Policy Gordon Research Seminar (Susan Cozzens and Nancy Gray, Gordon Research Conferences)

26. TLS: Assessing the Impact of Science Policy on the Rate and Direction of Scientific Progress: Frontier Tools & Applications (Jeffrey Furman, Scott Stern and Fiona Murray, National Bureau of Economic Research)

Measuring & tracking science and innovation:

27. Using Researcher Profiles to Demonstrate the Impact of Investments in Science (Griffin Weber, Harvard University)

28. Connecting Outcome Measures of Entrepreneurship, Technology, and Science (COMETS) (Lynne Zucker and Michael Darby, National Bureau of Economic Research)

29. The Impact of Research and Development on Quality, Productivity, and Welfare (Amil Petrin, University of Minnesota Twin Cities)

30. Career Dynamics in the Science and Engineering Workforce ( Catherine Weinberger, University of California Santa Barbara)

31. Investing in Science, Research and Technology: Where is the Biggest Bang for the Buck? (Sandy Dall’erba and Jaewon Lim, University of Arizona)

32. EAGER: Unified categories for describing and quantifying scientific research (David Newman, University of California-Irvine)


1. Tracing Influence and Predicting Impact in Science

Researchers: James Evans and Andrey Rzhetsky, University of Chicago

Abstract: This project develops powerful new measures and models that assess and predict the impact of academic research on science, technology, and the public. The evaluation of research has historically relied on quantity as a proxy for quality: progress is inferred from the amount of research produced and attention garnered, the number of articles is tallied, and academic citations and media mentions are summed. These quantities are cheaply measured, but cannot capture how fundamental a contribution is or identify the nature of its impact. Such nuanced evaluation of research quality demands that new claims be considered in the context of previous work.

Intellectual Merit: Building on progress in the measurement of scientific and technological novelty, as well as phylogenetic models of evolution, advances in computational language understanding, and the increasing electronic availability of historical and contemporary scientific texts, this project creates computational tools to identify scientific claims, embed them in their historical, conceptual, and geographical context, and thereby provide a multidimensional evaluation of the nature and scope of their impact. Claims extracted from text are complemented by article citations and unpublished expert opinions about paths of scientific influence, elicited through newly developed interactive online games. Taken together, these data provide the input for probabilistic models that exploit patterns in the structure of scientific language, the graph of citations, and expert opinion to assess the evolutionary and ecological importance of particular concepts and claims, articles and journals, scientists and institutions within the unfolding network of scientific and technical innovation.

These rich models provide the nuanced assessment of research quality and impact needed by scientists and policy makers at all levels, from individual scientists choosing a new research focus to departments making tenure decisions or federal agencies setting funding priorities.

The methods and models developed by this project are relevant to scientists and policy makers in many areas. The initial focus, however, is on the broad field of chemistry and its diverse, overlapping subfields, from analytical, physical and organic chemistry to environmental and agricultural chemistry, pharmaceuticals and toxicology.

Broader Impact: This project enables the creation of high resolution, dynamic maps of scientific influence in several broadly related fields. In education, such maps can be made interactive and serve as a teaching tool to help students understand the contributions and forces that have shaped their field of science. These maps also facilitate the social analysis of scientific production with a new level of precision, while orienting researchers and policy makers to the forces that shape scientific importance and technological promise. Most importantly, these influence maps provide the foundation for a collection of sharp measures that can identify different types of importance and the benefits and risks associated with each so that they can be assessed and balanced by researchers and policy-makers. By clarifying what is published and where, and by tracing the conceptual careers of scientists and inventors, this research offers insight into the factors that guide and deflect scientific attention and how these factors can be harnessed to achieve the greatest impact from public investments in science and technology.



2. The Long-Term Regional Economic Impacts from Public Investment in University Research

Researchers: Shawn Kantor and Alex Whalley, National Bureau of Economic Research

Co-Fund: Economics

Abstract: The goal of this revised project is to create a new database of county-level productivity of manufacturing firms from 1947 to 1992. These historical data have been published in the Manufacturing Census reports, but are not yet electronically available.

Intellectual Merit: This data-collection project will provide the first electronically available information on U.S. manufacturing at the county level over a fairly long period of time. The data will provide wide benefits to the scholarly community interested in the post-war economic development of the U.S., especially at the local level.

Our ultimate goal is to construct a body of data that will enable us to examine how publically-funded scientific knowledge flows across locations, and how these knowledge flows affect the development of regional economies. The data collected under the auspices of this grant will facilitate a study of how Space Race-science impacted regional economies from 1947 to 1992.

Broader Impact: Our research into this important historical episode in the ascendency of the modern American research university will help to uncover whether the broad economic effects of public investments in university science last for many decades or are simply transitory. They will also help to shed light on the channels through which university knowledge benefits the private manufacturing sector.



3. Organizations and the Diffusion of Scientific Knowledge

Researchers: Scott Stern and Michael Bikard, Massachusetts Institute of Technology

Abstract: The contribution of scientific research to economic growth is hardly controversial. Yet little is known about how scientific knowledge actually translates—or fails to translate—into economic gains. Historical examples of advances in science reveal that while some discoveries are commercialized right away, others remain unexploited for years, even decades, until their commercial potential is understood. Discerning why some discoveries get commercialized and others do not is very difficult, however, because the commercial value of the new knowledge cannot be ascertained unless commercialization actually occurs.

Intellectual Merit: Do scientific discoveries have the same economic impact if they come from a university or from a firm? What are the characteristics of these academic or industrial organizations that would affect its impact in the marketplace? This project takes on the challenge of these questions by introducing a novel empirical strategy. It uses simultaneous discoveries to conduct the first "twin study of new knowledge." This method allows the researcher to compare the commercialization or non-commercialization of the same new knowledge emerging at the same time in two distinct environments.

The project, which is divided into two parts, draws on interviews and archival data to study the overall patterns of how new knowledge is harnessed for commercial value and the mechanisms behind these patterns. It takes advantage of the applicant's original, pre-existing dataset of automatically and systematically identified simultaneous discoveries. Part I examines whether scientific findings are "trapped inside the ivory tower" i.e. whether they are less likely to be commercialized out of a university than out of a firm. Part II constructs a detailed analysis of a small number of cases of simultaneous discoveries. This allows for a more in- depth inquiry into how different types of organizational arrangements influence the prospects for commercialization.

Broader Impact: This research has immediate practical implications for the Science of Science and Innovation Policy as it will uncover the extent to which scientific knowledge that could be commercialized remains unused on laboratory shelves. Specifically, it increases our understanding of the role of different types of organizations in diffusing new knowledge. For instance, scientific collaboration, in addition to fostering the circulation of ideas, might increase the likelihood that new knowledge will be commercialized. There are immediate implications as to the conditions under which collaborative research ought to be supported. By encouraging organizational structures that foster the commercialization of scientific knowledge, SciSIP might be able to decrease the number of unused scientific discoveries.

This interdisciplinary research is also of direct interest for economists, organization theorists, and sociologists of science given its insights into knowledge spillovers, the management of technological innovation, and competition in science. In addition, this research improves our understanding of strategies that business practitioners can implement in order to commercialize new scientific knowledge successfully. By investigating and breaking down exactly how organizations use and diffuse new scientific knowledge, this research enables a better management of knowledge commercialization, accelerating the spread of science into the economy, and increasing the economic impact of scientific research.



4. Incubators of Knowledge: Predicting ProtÉgÉ Productivity and Impact in the Social Sciences

Researchers: Cassidy Sugimoto, Ying Ding and Stasa Milojevic Indiana University

Abstract: Doctoral education plays a pivotal role in shaping the careers of future scholars and, thereby, in making an impact on the trajectory of knowledge creation in a nation. During a doctoral program, students are acculturated to the norms of the discipline, learning scholarly practices and behaviors that guide them for a lifetime. Advisors, as guides to these scholarly journeymen, serve as critical gatekeepers to the discipline and can have a profound influence on their doctoral students. Doctoral students comprise a larger portion of the academic workforce, yet scholars have very little knowledge of their place in scholarly networks, the degree to which they contribute to scholarly output, and the impact of this output. Very little quantitative analysis shows the relationship between advisors' scholarly practices and the future success of their advisees.

Intellectual Merit: This study investigates these issues from two main angles: understanding the contribution of doctoral students to social-science research (the extent and character of this contribution) and the impact of this research (visibility through citations); and examining the advisor's knowledge base and knowledge- diffusion practices, and whether these factors are involved in expanding knowledge frontiers and how they relate to the career trajectories and future success of doctoral students. This approach allows quantification of advisor behaviors and documentation of patterns within advisee behaviors. This research thus produces a viable framework for predicting advisee success based on advisor qualities and individual students' publication practices in the course of their doctoral studies. This predictive model can then be used by science policy makers and administrators for more efficient allocation of resources and to identify ways to promote innovation in higher education. This work supports a quantitative-based understanding of contribution of doctoral students to the creation of knowledge and of the relationship between the scholarly practices of advisors and the productivity and impact of their protégés. Such a model will also enhance our understanding of how the different scientific practices of the advisors -- including their embeddedness in particular disciplines, degree of mobility among different disciplines, and involvement in highly collaborative interdisciplinary work -- will have an effect on the career trajectories and scientific success of their advisees.

The combination of multiple datasets and the innovative analyses in this first research study on protégés provides a rich foundation for the development of new metrics and evaluations of knowledge diffusion, scholarly productivity, and scientific impact in doctoral education.

Broader Impact: Educational opportunities are provided through the funding of students and the integration of this project into the classroom. In addition, this research provides a platform for future analyses on the interaction of mentoring in doctoral education with individual protégé characteristics. The research products of this work will be disseminated at national and international conferences and workshops. The process of matching heterogeneous datasets, as well as the datasets themselves, will be detailed and made available online in order to enhance replicability and enable other scientists to adopt and expand these approaches.



5. The NIH public access policy: Potential Impact on Physicians and Community Health Organizations

Researcher: John Willinsky, Stanford University

Abstract: In 2008, the National Institutes of Health (NIH) instituted a Public Access Policy that requires recipients of NIH funding to make all resulting peer-reviewed journal articles publicly accessible within a year of publication. This amounts to an estimated 88,000 articles annually, with similar policies in other countries leading to a significant increase in public access to biomedical research. This study assesses whether physician and community health organization access to published research can be expected to make a difference in the services they provide.

Intellectual Merit: The sample for the study involves two professional communities, physicians and community health organization (CHO) staff. The approach taken is to enroll a sample of 300 physicians in a yearlong Randomized Controlled Trial that measures information-seeking behaviors in clinical and other settings. The experimental group has access to the journal collection of Stanford University Library, as a proxy of future public access and a point-of-care research summary service (UpToDate). The control group has access to UpToDate, as well as the current level of public access, principally to article abstracts and that proportion of articles that are open access (20%). The second phase surveys 50 CHO staff on their research use and information skills. Those who opt for training in using PubMed are provided with yearlong access to Stanford's journal collection.

Participants are then debriefed about their research use, and a secondary analysis determines demographic, technical, and training factors that affect information access and utilization. This study brings a high standard of evidence to bear on assessing how professionals discover, access, and utilize research in clinical and public health practice, as well as for personal development. The study demonstrates the value of mixed methods research by combining the results of both gold-standard precision (re: use of research in practice over time) and detailed qualitative understanding (re: how research is used). The findings should inform policy initiatives in areas of information access and literacy, evidence-based medicine (EBM), and medical education, while the inclusion of CHOs increases their relevance to the healthcare needs of underrepresented groups.

Broader Impact: The NIH Public Access Policy is part of a broader international "opening" of science (e.g., open access, open source software, open data, open educational resources). This has led to Congressional initiatives to extend the policy across government agencies, as well as to pushback that would curtail any such initiatives. Thus, the need to assess the value of such access to health care providers to see if it has the potential to contribute to evidence-based care, a higher quality of treatment, and reduced health-care costs through the use of the latest research. At issue is whether public and professional access to federally funded research has the potential to improve the practices of related bodies of professionals, as this has a bearing on the spread of such policies, the design and delivery of such information, the curriculum in medical education, information literacy education in other professions, and expectations for the public use of online information more broadly.



6. The Evolving Research Enterprise

Researchers: Maryann Feldman and Michael Roach, University of North Carolina at Chapel Hill

Abstract: Private foundation funding of academic research has grown substantially over the past 10 years. Increasingly, foundations employ a new strategic funding process modeled after venture capital investing, in which project outcomes are identified by the sponsor with a specific problem to be solved, a team is selected based their potential contribution and a community of common inquiry is created and funded over a long period of time. This new funding model originated with disease-oriented foundations, such as Michael J. Fox and Faster Cures and is diffusing to other types of foundations and agencies within the federal government, suggesting that this new model may offer advantages to scientific inquiry.

However, despite the growing importance of foundations and strategic funding models, little is known either about whether foundations complement or substitute for other sources of founding or whether strategic funding practices can enhance the overall functioning of the American research enterprise. This study considers the role of private foundation funding in the larger university research enterprise as well as the impact -- both in terms of research practices and research outcomes -- of this new strategic funding model. This project describes how the university research enterprise is evolving in response to the demands of new funding sources coupled with increased institutional fiscal pressures and calls for greater problem solving relevance.

Intellectual Merit: This project uses detailed records of research proposals, funded projects, and intellectual property (IP) agreements from a set of diverse research universities to make three intellectual contributions. First, it examines the extent to which private foundation sponsorship provides a new model of research funding that affects the conduct of research. Second, it compares the contractual agreements of foundations employing strategic practices to managing funded projects to the contractual research agreements of other sponsors to assess the extent to which a new open model of innovation is being implemented. Finally, it evaluates the extent to which foundations, in general, and venture philanthropy in particular, complement or substitute other types of funding sources.

Broader Impact: The current omission of foundation and nonprofit funding presents a confounding effect that would introduce downward bias in the estimation of the impact of public funding. This study expands the discussion of the funding of academic research to recognize the role of foundations and further to consider differences in funding strategies and the use of contractual terms in sponsored research agreements.

Recognition of this richness provides evidence useful for measuring the impacts of research funding and the ongoing debate about the efficacy of different research funding strategies.



7. Managing Community: The Organization and Management of Federal Research Funding Agencies

Researcher: Michael Piore, Massachusetts Institute of Technology

Abstract: This project investigates the organization and management of three Federal agencies that support scientific research to identify the key differences among the agencies and how they impact the type of research that is conducted, the development of communities of interest, and research outcomes. Preliminary research suggests there is significant variation in understanding, within both the agency personnel managing the funding process and the scientific community of researchers that receive funding, of what critical factors define each agency and influence research outcomes. The project examines and compares funding activities at the National Science Foundation (NSF) and the Defense Advanced Research Projects Agency (DARPA). It also includes an exploratory component focused on the National Institutes of Health (NIH) and gathers some observations about other Federal funding agencies that have supported NSF-, DARPA, and NIH-funded researchers.

Intellectual Merit: A major contribution of this research is to advance understanding of the impact on funding decisions of the role of the program manager (or project manager), the degree of decision-making autonomy at this level of the organization, and the way in which that autonomy is managed and directed. The project draws heavily on the literature on "street-level" bureaucracy in the public sector and on the decentralization of authority in large corporate organizations of the private sector. In this, it departs from the emphasis in the recent literature on the management of the public sector by recognizing the role of the norms of different communities (e.g., the scientific community; the professional community of program managers, the military community) and the way in which the evolution of these norms can be guided and directed by various management approaches. The research is based on interviews with current and recent funding agency staff, and with researchers who have applied (successfully or unsuccessfully) to the agencies for funding. It also draws on a confidential database from the Massachusetts Institute of Technology on all applications for funding submitted within a fifteen year period, linked to data on patents, licensing and scholarly publications related to federal funded research projects.

Broader Impact: The project is designed to help policy makers and administrators evaluate the design of federal research programs, understand how they relate to each other in the eyes of the researchers who they fund and whose behavior they seek to influence, and better identify the levers for effectively managing research and development. It will also provide a preliminary model for attempting to evaluate the impact of scientific investments. Finally, it provides recommendations regarding ways to improve managerial performance, an analytical template for programs beyond those that are the direct focus of the study, and contribute to the literature on public sector management more broadly.



8. Managing Community: The Organization and Management of Federal Research Funding Agencies

Researchers: Erica Fuchs and David Hounshell, Carnegie Mellon University

Abstract: This project analyzes existing U.S. consortia models, in particular, at the Semiconductor Research Corporation, to inform governance of and success metrics for the U.S. National Network for Manufacturing Innovation (NNMI). It prepares insights in time to inform governance and success metrics for the Pilot Institute for Manufacturing Innovation just launched on May 8, 2012 and the 2013 expansion of NNMI.

Intellectual Merit: Past research has consistently found a positive impact of consortia on the research productivity of participants in the technological areas they target. This research, however, has largely failed to provide insights into successful consortium governance. Arguably one of the most famous examples of a government-industry research consortium, and one with widely-touted success, is SEMATECH. The example of SEMATECH, however, leaves much to be desired. Perhaps most importantly, its focus is primarily on short- term (1-3 year) R&D in the context of vertical collaboration to support the advancement of equipment suppliers. Thus while successful on certain axes, SEMATECH fails to provide an example of collaborative R&D for long-term horizontally-collaborative research efforts; as a matter of fact, in this specific context, SEMATECH provides an example of early failure. And yet, it is precisely this type of horizontal coordination of long-term technology development that may be particularly crucial to advancing innovation effectively across a multitude of institutions in a vertically disintegrated supply chain, as can be critical when introducing emerging technologies into established industries and helping them develop industries of their own -- issues central to revitalizing U.S. manufacturing.

Broader Impact: With the decline of corporate R&D labs and the globalization and vertical disintegration of the supply chain, the United States innovation ecosystem faces unprecedented challenges in maintaining its economic and technological predominance, increasing jobs throughout the economy, and supporting long- term technical and economic growth. For these challenges to be met, managers in industry, government, and academia must find ways to work hand-in-hand toward meeting the common challenges facing each of them as well as the nation as a whole. In this new environment, research consortia (as intermediate forms of organization, between a single firm and the overall market) hold tremendous potential opportunities to achieve improved innovation policy outcomes.



9. A Comparative Study of Structural Influences on User-Engaged Ecology Research

Researchers: Mark Neff, Allegheny College

Abstract: This project investigates the impacts on scientists’ ability to conduct user-engaged research of two important but relatively poorly-understood types of science policies: university publication requirements that score authors based on publication in particular journals, and the editorial preferences of those journals. When researchers are strongly incentivized to publish in particular journals, the editorial boards of those journals may have significant influence over the types of research conducted by those researchers. This research uses a comparative ethnographic approach to understand the impacts of these policies across five national settings and will yield best practices in publication requirement policies for institutions that wish to encourage user- engaged ecology research.

Intellectual Merit: An emerging literature suggests that when scientists interact with potential users of their knowledge throughout the research process, from problem formulation to result dissemination, they are best able to contribute solutions to real-world policy problems. Conducting “user-engaged” research, as these authors term it, ensures that the resulting knowledge is relevant to the problems at hand and produced on an appropriate timeline. Additionally, knowledge users are more likely to see the resulting information as credible, legitimate, and salient to their purposes. The level of interaction with knowledge users required by this approach would represent a substantial shift in the way most academic ecologists conduct their research, and it is not immediately clear the extent to which scientific institutions would permit these changes.

Broader Impact: The knowledge generated by this project will help federal and state science-funding organizations, universities, journal editorial boards, and other institutions and individuals create incentives such that researchers are able to conduct user-engaged science, when appropriate. This work will contribute to the effective use of public research money toward the goals for which it is allocated. Additionally, this project will train undergraduate researchers in document analysis, translation, semi-structured interviewing, and related research methods, thus preparing them for advanced science policy scholarship.



10. ICES - GMU Workshop on Internationalization and Competitiveness

Researchers: Eskil Ullberg and Daniel Houser, George Mason University

Co-Fund: Office of International Science and Engineering

Abstract: This award supports a workshop (on internationalization and its competitive impacts) to be hosted at George Mason University in Arlington VA in early 2013, responsive to the Dear Colleague Letter (12-010) on Describing the Conduct of Science in the Information Age.

Intellectual Merit: The workshop is organized around two academic themes: the measurement of the extent of internationalization of research groups (university departments, corporate R&D facilities, etc.) and the linkages between these measures of internationalization and the competitive performance of the research groups' sponsors (e.g., the results of university technology-transfer departments, overall company performance, etc.). Although both internationalization and performance are justifiably seen to be generally desirable, both workshop themes, of the proper metric of internationalization and the competitive implications, are important and unstudied at the level required to make scientific policy recommendations. Basic and easy-to-collect measures, such as the percentage of a given patent's named inventors who were born outside the USA, do not capture the international collaborations among multiple research groups that lead to truly breakthrough innovations.

Broader Impact: Linking this measure of internationalization with competitive performance is an essential precursor to creating innovation policy related to internationalization in the US and elsewhere; while a number of social-science theories predict a generally positive statistical relationship between international collaborative activity and competitive performance, measurement difficulties on the independent internationalization variables have made it difficult to make policy recommendations based on the simple correlational studies done to date. The workshop's invitees combine capabilities for metric creation and evaluation, expertise on the intellectual property development process, international collaborations, and the causal drivers of competitive performance.

The international component of this workshop, in addition to the focus on the competitive implications of internationalization, includes direct collaborations with research colleagues in Sweden. The Office of International Science and Engineering (OISE) provided support for this international component and joint support for the involvement of junior faculty members and doctoral candidates.



11. Empirical Studies of Innovation in Health Care Markets

Researcher: Heidi Williams, National Bureau of Economic Research

Abstract: In 2001, Nature and Science published the initial sequences of the human genome. In the decade since, the sequenced human genome has dramatically accelerated biomedical research, and has laid the groundwork for a revolution in medicine. Yet Nature's editorial page in 2001 was focused not on the scientific and medical promise of the sequenced genome, but rather on a controversy over rules governing public access to genomic data. Indeed, by many accounts biomedical researchers frequently struggle to navigate a complex landscape of non-scientific obstacles to scientific research and product development - such as negotiating access to patented and non-patented materials. Successfully translating the science of genomics into improvements in health requires resolving these economic and institutional obstacles.

Intellectual merit: This project outlines a long-term agenda of research and education investigating barriers to innovation in health care markets, with a particular focus on the science and medicine of genomics. The project consists of three parts. First, it constructs a series of new datasets to measure the development and diffusion of genomic technologies, and develops open-access programs to enable others to replicate, update, and extend this data construction. Second, this project incorporates these data into a series of research projects to test and evaluate theories about economic factors that may be hindering innovation in biomedical science and health care markets. The direct goal of these studies is to shed light on specific questions, such as whether gene patents are hindering scientific research and product development. The broader goal is to develop new research designs that can subsequently be applied to other markets. Third, this project develops a monthly research workshop focused on the goal of integrating PhD students into this agenda. The goal of this workshop is to build a laboratory - generating leads on new research questions, and supporting students in collecting data and designing empirical strategies to test new hypotheses.

Broader Impact: Barriers to innovation in the health policy context have important consequences for human health and welfare. In addition, for federal and state governments, rising health care costs constitute the principal challenge of fiscal policy in the coming decades. The best available estimates suggest 30 to 40 percent of health spending - more than half a trillion dollars per year - could be eliminated with no adverse health impacts. Identifying barriers to medical innovation holds the potential to encourage the development of new technologies that can reduce health spending while also improving health outcomes.



12. EAGER-ORCID: Investigating ORCID as an accelerator of science of science policy

Researcher: Ian Foster, University of Chicago

Abstract: The ability to attribute the results of scholarship to individual scholars is a fundamental enabler of a rigorous and data-driven science of science policy; it enables the automated linkage of research inputs to research outputs in an accurate and consistent manner. The research explores whether a new researcher identification and profile system can provide the basis for an automated attribution system.
The goals are to refine and deploy a profile system by working with a small number of US research institutions and the federal government to explore how this system works in practice when used to acquire and provide access to information about US researchers. This research is suitable for the EAGER program because it is simultaneously high risk (there are many ways in which a researcher identity system can fail, including technical, institutional, and sociological) and high reward (it can fundamentally change the way in which scientific activity is documented).

Intellectual Merit: The attribution problem is extremely complex. There are technical challenges such as how to generate unique researcher identifiers, match those identifiers with research outputs such as publications and patents and maintain large numbers of researcher profiles in a reliable and efficient manner. There are human computer interface issues, such as privacy concerns and user-friendly claiming mechanisms. There are also substantive social issues such as incentivizing and rewarding participation as well as achieving network effects.

Broader Impact: The researcher identifier and profile system to be developed and evaluated in this project has benefits well beyond science policy. Researcher profiles can be used to streamline grant application, publication manuscript, and employment application processes. Researchers could use the approach to include all their contributions to the scholarly record, such as peer review, data curation, and software development. By making data about research and researchers more visible, profiles can also help researchers locate potential collaborators and develop networks. Research institutions can use such profiles to help them to evaluate the research outputs associated with specific research teams, departments, and/or institutions, and to support the job recruitment process. In all cases, profiles will enable the identification of strong areas of research and to track the publications of their faculty. Scholarly societies could use profiles to enhance their public member profiles. Publishers could use profiles to track authors and reviewers in their journal submission systems, with the ability to screen effectively for conflicts of interest. Reliable linkage among articles by the same authors and their collaborators could help promote the discovery of related scholarly works.



13. Atlanta Competitive Advantage Conference PHD Student Workshop

Researcher: William Bogner, Georgia State University

Abstract: The objective of this project is to expand and enrich the pre-conference Research Development Workshop (RDW), which is held prior to the annual Atlanta Competitive Advantage Conference (ACAC). Emory University, Georgia State University and The Georgia Institute of Technology jointly sponsor ACAC. ACAC will be holding its 9th Annual Conference on May 14th-17th of 2012.

Intellectual Merit: ACAC's mission is to provide a unique forum for the presentation, discussion and encouragement of research and theory development in the fields of industrial organization economics, organization theory, entrepreneurship, and strategic management.

Broader Impact: This proposal to expand the RDW greatly increases the emphasis at ACAC on Ph.D. student research. The proposed expansion of ACAC's RDW in 2012 will triple the number of Ph.D. students that participate in the RDW compared to 2011. It will also increase by 50% the number of senior scholars in the fields of industrial organization economics, organization theory, entrepreneurship and strategic management who will mentor the Ph.D. students and double the number of hours of the RDW, enabling these enlarged groups of students and scholars to work a significant amount of time on a 1:1 basis. The supported Ph.D. students will be able to attend and participate in the main conference as well.



14. Inter-Industry Differences in the Antecedents and Censequences of Industrial Scientists Mobility and Entrepreneurship Decisions

Researchers: Rajshree Agarwal-Tronetti and Seth Carnahan, University of Maryland College Park; Martin Ganco, University of Minnesota Twin Cities; and Benjamin Campbell, Ohio State University

Co-Fund: Science of Organizations

Abstract: This research examines inter-industry differences in the antecedents and consequences of industrial scientists' mobility and entrepreneurship decisions. Three questions are analyzed 1) Why do some high technology industries have higher rates of scientist mobility and entrepreneurship than others?; 2) What are the innovation diffusion patterns across high technology industries due to mobility and entrepreneurial entry of scientific personnel?; and 3) How do industry characteristics interact with firm and individual attributes to impact employment growth of new ventures created by industrial scientists?

The project centers on the role of complementary assets and posits that inter-industry differences in their importance play a critical role in explaining industry heterogeneity in the rates and outcomes of industrial scientist mobility and entrepreneurship decisions and thus serve as the theoretical underpinnings for all three research questions. A particular focus is on inter-industry differences in the importance and transportability of complementary non-human assets, how they shape the career decisions of individual scientists, and how they affect both the levels of scientist mobility and entrepreneurship within industries and the direction of scientist mobility and entrepreneurship that occurs across industry boundaries. The empirical analysis of the data focuses on scientists working in high-tech industry settings tracks scientists' entrepreneurship and mobility within and between these high-tech sectors.

Intellectual Merit: In contrast to most of the prior literature that studied the antecedents and consequences of the career choices of scientists at the firm level and within a single industry context, this research examines the questions by focusing on individual level decisions in a broad cross-industry setting. This thus examines multiple important gaps that are of interest to both policy makers and academics. In particular, by building the micro-foundations of the knowledge diffusion process at the individual scientist unit of analysis, it helps to build a systematic understanding of inter-industry differences in industrial scientist mobility and entrepreneurship in high tech industries. Understanding industry heterogeneity in the antecedents and consequences of the career choices of industrial scientists is particularly important in the context of understanding and promoting industry renewal as scientists are likely to play a key role in the creation and commercialization of new technologies which can have important growth effects at the firm-, industry-, and regional-level.

Broader Impact: The study informs science and innovation policy makers about what industry conditions are most conducive to employment growth via entrepreneurship by scientific personnel. In addition, it examines the results of incentivizing a potentially crucial but heretofore under-examined source of high tech entrepreneurship: scientists employed outside of the focal industry. The results may also inform scientists' career decisions about industry-level factors that may facilitate or create impediments to successful start-up entry and subsequent growth.



15. NSF Patent Data Workshop

Researchers: Alexander Oettl, Georgia Tech Research Corporation

Abstract: This workshop facilitates the interaction of researchers working with patent data for the purpose of scholarship on innovation policy. The focus is on advancing and sharing knowledge on a series of technical issues including name disambiguation and matching patent data to other economically-relevant data sources.

Intellectual Merit: The workshop provides a unique forum for the exchange of patent data measurement related topics with the focus of informing innovation policy in a multi-disciplinary environment. It reduces unnecessary duplicative effort on data, measurement, and methodology, and hence increases the capacity to build both theory and for innovation policy. The workshop also advances an important SciSIP goal of creating a bridge between the academic and practitioner communities by directly engaging the United States Patent and Trademark Office (USPTO) and the academic community through the workshop participation of the USPTO's Chief Economist.

Broader Impact: The workshop advances the establishment of a robust data infrastructure for cumulative, transparent, and high-quality research and the ability to translate that research. It is an important step in continuing the empirical research tradition that has been built around the economic consequences of innovation. It provides an environment for high-quality high-impact research, with an emphasis on measurement. It fosters the development of a research community and research norms, and facilitates cumulative research across this community.



16. Understanding Innovative Science: The Case of the Wisconsin Institutes for Discovery

Researchers: Daniel Kleinman, Gregory Downey and Noah Feinstein, University of Wisconsin Madison

Co-Fund: Science, Technology & Society

Abstract: What is distinctive about contemporary science? This question is addressed in this project by examining how emerging ideas and social arrangements of contemporary science affect the practices and outcomes of scientific research in one strategic case, the Wisconsin Institutes for Discovery. These institutes are research organizations that explicitly seek to break disciplinary barriers, forge new tools for public/private collaboration, and speed the application of transformative science.

Intellectual Merit: The project will engage in this analysis by using multiple methods including ethnography, oral history, and archival analysis to engage in historical and contemporary comparisons. It will explore how the Wisconsin Institutes for Discovery embody what is different about contemporary science as well as what remains the same. Three questions guide the research:

1. How do widely discussed ideas about such issues as interdisciplinarity, transdisciplinarity, and public/private collaboration shape the evolution of a new institution?

2. How do these ideas affect scientific work as well as the public work of forging relationships with the Institutes' constituencies?

3. How (and for whom) do new ways of doing science succeed? What new metrics are needed to capture their success?

Broader Impact: The project will support graduate research training and scholarly publication, expand a publicly accessible electronic archive, enabling others to study the Wisconsin Institutes of Discovery. Broad dissemination of findings will be accelerated through a concluding symposium that brings together the researchers, stakeholders in the Institutes, science policymakers, and invited scholarly commentators. A policy- oriented whitepaper and a book summarizing the results of the symposium will also serve to broaden the projects overall impact.

The results of this project will have the potential to guide future investment in research and research infrastructure. It will add new evidence to ongoing discussions about the value of interdisciplinarity, the pros and cons of public/private collaborations, and the community-wide effects of centers and institutes.



17. Evaluating the Effect on Cyberinfrastructure on Universities Production Process

Researcher: Amy Apon, Clemson University

Co-Fund: INSPIRE: Integrated NSF Support Promoting Interdisciplinary Research and Education, EPSCOR: Experimental Program to Stimulate Competitive Research, Office of Cyberinfrastructure

Abstract: This project undertakes an interdisciplinary and novel approach to the problem of measuring the effects of investment in cyberinfrastructure to universities' production processes of research outputs and vital educational services. A decision to support funding of the infrastructure that supports research, or a decision to support funding of focused research activities, is an increasingly critical decision with far-reaching impacts not only to the institutions receiving those funds, but also to national competitiveness. While it is generally agreed that cyberinfrastructure is essential to scholarly inquiry in some science fields, the scope of cyberinfrastructure's broad effects on the growth of knowledge, to the academic enterprise, and to areas of science has not been explicitly quantified.

Intellectual Merit: This work extends the state of knowledge of frontier efficiency analysis (FEA) techniques, a rigorous statistically-grounded approach, and uses FEA in a novel application to examine the returns to cyberinfrastructure investments in research institutions. Such work is strongly interdisciplinary: experts in econometrics will be working collaboratively with experts in computing and cyberinfrastructure to apply and extend the state-of-the-art in data management, extend our developed Unified Data Framework, and prepare and curate a significant new body of data of great interest to decision-makers in cyberinfrastructure investment and science policy. This research also makes a contribution to statistical estimation theory, developing new central limit theorem results applicable to means of nonparametric frontier efficiency estimators, permitting testing of hypotheses regarding returns to scale and other aspects of universities' productivity. To overcome the problem that nonparametric frontier efficiency estimators are biased in finite samples and have slow convergence rates that depend on dimensionality of the particular problem, this project uses subsampling ideas to construct new statistics for hypothesis testing in nonparametric frontier efficiency estimators for which central limit theorem results can be obtained.

Broader Impact: Finally, the project makes a dual contribution to science and innovation policy and to cyberinfrastructure investment decisions, using frontier efficiency analysis (FEA) techniques to evaluate the productivity of research institutions who have received various amounts of cyberinfrastructure investments. Project personnel will collect and curate a large body of data on educational and research institutions' productivity and performance as a part of this analysis. The analysis of this data will allow examination of important science-policy questions on investment in cyberinfrastructure. The project will support 50% of a postdoctoral associate and a full- time Ph.D. student in interdisciplinary econometric and computer-science research. The techniques and tools utilized in this research are to be incorporated into a newly-developed course in Data-Enabled Science, which, over the course of the project, is expected to reach over 40 students in a variety of science and engineering disciplines.

The overall project is potentially transformative to the science of science policy: the project's development and demonstration of the use of formal measures for capturing the effectiveness of investment has the potential to significantly change the processes used to construct portfolios of funded scientific-research projects, and to add significant quantitative support to the policies by which these decisions are made. The specific case study (the effect of cyberinfrastructure investments on research and educational outcomes) can immediately inform institutions, states, and governmental funding bodies about opportunities for high-priority future investments in this area, particularly by identifying overlooked opportunities where modest investments will yield significant returns.



18. Sensible Science: A Sociometric Approach to Collaboration in Synthesis Groups

Researcher: Edward Hackett, Arizona State University

Co-Fund: Directorate for Biological Sciences; Science, Technology & Society

Abstract: This study examines how interdisciplinary groups of scientists collaborate to produce explanations that are highly integrative and original. This intellectual process, which is called synthesis, has the potential to transform fields of science and to address challenges concerning environment, energy, and other matters. It does so by integrating theories, methods, or data across disciplines, professional sectors, or spatial and temporal scales to produce explanations that are deeply original and exceptionally broad in applicability.

Synthesis is important to science and society, but the organizational and social conditions necessary to achieve it are not well understood, and so the ability to manage them is limited. This project uses sociometric sensor badges, a new research technology for the study of small group behavior, in combination with well-established methods in social studies of science, small-group research, sociology, and organizational behavior, to examine how patterns of group organization and interaction influence performance and creativity.

Intellectual Merit: Studies of small groups, particularly groups of scientists, tend to rely on observation. Valuable as this is, observation is necessarily limited by the observer's attention or focus and tends to produce qualitative data. Sociometric sensors can complement observation by revealing new facets of group dynamics and measuring them precisely and unobtrusively, and can also tap the affective dimension of collaboration.

Sociometric sensors, and the new insights into group behavior they will produce, may transform studies of collaboration, synthesis, and emotion in science, and advance research in related fields.

Broader Impact: Interdisciplinary synthesis and innovation are vital for addressing societal problems, and policies to encourage synthesis are high on the national science agenda. By improving our understanding of synthesis and our methods for studying it, this project will lay a strong empirical foundation for better policies, decisions, and organizational practices concerning collaboration, interdisciplinarity, and synthesis.



19. Incentives for Researcher Profile Maintenance and Access, and Their Value in Science and Innovation

Researchers: Erik Lium, Claire Brindis, Mini Kahlon and Tuhin Sinha, University of California San Francisco; and Ian Foster, University of Chicago

Abstract: The increasing complexity of science means that it is increasingly important to pull together teams in order to produce high quality research. This means that it is important to develop technologies that identify researchers with the right set of skills so that such teams can be expeditiously formed. This is particularly true in biomedical science, where the translation of research into clinical care often requires the expeditious convening of the right group of researchers, clinicians, and consumers. A variety of different technologies have evolved so that scientists can profile their skills; however it has been extremely difficult to incentivize researchers to adopt and use these technologies. This research explores the specific value that academic profiles may provide, how that value may be maximized in these larger research coordination networks, and what motivates researcher adoption and use.

Intellectual Merit: This research advances understanding of what features of profiling tools can be used to incentivize the scientific community to generate social capital in the sciences. Many examples of features are prevalent on the social networking sites that are currently popular on the Web. For instance, Facebook's initial success was attributed to incentivizing features such as exclusivity, relationship status, poking mechanisms, and directory services. For Twitter, the incentivizing feature is the availability of diverse usage channels (professional/private) and multi-client applications. The incentivizing feature for LinkedIn has been the development of professional networks and discourse only. Almost nothing is known on what incentivizes scientists. This research fills the gap by collecting an unprecedented set of data in both a biomedical and a multidisciplinary environment on what incentivizes profile adoption, influences self-curation, and increases the social capital of profiles through research networking tools. It assesses and disseminates the impact of using the data sources on the accuracy and comprehensiveness of data matched to investigators. It assesses the impact of auto-population, new data sources, and value-added features on usage and engagement. It also assesses the impact of tools on industry-academic alliances through the assessment by survey and focus groups of those actively engaged by the University of San Francisco's Office of Innovation, Technology and Alliances

Broader Impact: Findings from this experiment can have direct impact on the progress and development of partnership methods and systems. The broad availability of well curated profiles should advance the quality of both science and innovation. The research advances understanding in all areas of science where collaborative efforts are required to maintain and progress the capacity for innovation.



20. Scientific Collaboration in Time

Researchers: Carl Lagoze, University of Michigan Ann Abor and Steven Jackson, Cornell University

Co-Fund: Science of Organizations

Abstract: This project explores an innovative interdisciplinary approach for studying the dynamics of scientific collaboration. Over the last decade, science policy has increasingly focused on the benefits of knowledge sharing, openness, and collaboration, a policy theme that has motivated a number of large recent investments by the NSF and other funding agencies in cyberinfrastructure development and deployment. The success of these investments however depends on attention to field-specific practices and cultures that may influence or even block adoption of cyberinfrastructure. This exploratory project sets out to advance a new interdisciplinary methodology that integrates ethnographic field studies with the analysis of large-scale publication networks to strengthen the empirical basis for understanding field differences in scientific collaboration.

Intellectual Merit: Methodologically, this project offers an innovative integration of quantitative network analysis of large publication networks with qualitative ethnographic field studies. Theoretically and substantively, it illustrates on the multi-level temporal dynamics that shape, and sometimes frustrate, efforts at scientific collaboration and innovation. The methodology developed in this project supports field-level comparisons of scientific communities and fills a critical gap in the research capabilities of scientometrics, innovation studies, and science policy, which are too often split between macro-scale (countries, disciplines, journals) or micro-scale (individuals, specific research localities) analyses, neglecting meso-scale dynamics that are formative to the shape and outcome of scientific knowledge production within and across fields.

Specifically, this project investigates the addition of a temporal dimension to a network ethnographic approach to study and compare temporal dimensions of scientific collaboration across research fields. The test cases are provided by two research fields (in ecology and at the boundary between physics and chemistry) that exhibit significant variation in the temporal rhythms that underlie the structure of short- and long-term collaborations within a research group and between this group and its collaborators. Adding temporality to the analysis increases the resolution of collective structures in publication networks to support the strategic sampling of ethnographic field sites; enables the study of the emergence and evolution of collaborative structures at the team, sub-community, and field level and the detection of field-specific historical trends; and supports the search for field differences in the timing and rhythms of collaborative activities. This EAGER projects serves to explore and test the suitability of a network-analytic approach to capture scientific collaboration dynamics at various scales before deploying it in larger-scale empirical studies.

Broader Impact: Through strengthening the empirical base to inform science and innovation policy and to guide investment decisions, this exploratory project develops a methodological approach with a wide re-use potential for the comparative study of collaboration dynamics in scientific communities. Understanding the field-specific dynamics, tensions, and challenges of collaboration practices and their evolution over time is critical when the potential benefit and impact of policy and socio-technical interventions (such as cyberinfrastructure, data sharing mandates, and new incentive schemes), and, by improving metrics to quantify the impacts of these investments, complements existing investments and informs future investments in these specific areas.



21. Science of Science and Innovation Policy: Principal Investigator Conference, 2007-2011 Awards

Researcher: Kaye Husbands Fealing, National Academy of Sciences

Abstract: In 2006, U.S. Science Adviser John H. Marburger, III called for a multidisciplinary approach to creating a solid evidentiary platform for science policy at national and sub-national levels, as well as the emergence of a field of science policy studies. Science and innovation policy practitioners have long had a keen interest in obtaining quantitative data and qualitative information that they could use to deliver more effective policy decisions. Researchers in several disciplines have for decades sought to push the boundaries of understanding various elements of the global ecological system of innovation. The NSF initiated the SciSIP program to encourage basic research and to further the development of a community of practice in this field. Since the first awards were made in 2007, the SciSIP program has funded more than 150 researchers and their graduate students. The program has also spawned the STAR METRICS program, a collaborative effort between the NSF and the NIH. The STAR METRICS program develops tools and mechanisms by which federal expenditures on scientific activities can be measured, with particular focus on quantifying productivity and employment outcomes.

The fifth year of the SciSIP program, therefore, is a critical milestone and an opportune time to showcase its research productivity and contributions to many long-standing questions regarding investment in and organization of science, engineering and innovation activities in the U.S. and in other nations. In keeping with the goals of the SciSIP program, the National Academy of Sciences convenes a two-day public conference to foster intellectual exchange among funded researchers and between these researchers and science, technology and innovation policy practitioners. The conference is intended to be the largest gathering of SciSIP principal investigators since the program's inception.

Intellectual Merit: The SciSIP principal investigators conference features invited presentations and discussions, as well as poster sessions. Topics highlight advances in the emerging field of the science of science and innovation policy. In particular, models, frameworks, tools, and datasets comprising the evidentiary basis for science and innovation policy are the focus of the event, and presentations by SciSIP researchers fall under several themes, including: return on investment models; organizational structures that foster accelerated scientific productivity; the roles of universities and government in technology transfer and innovation; technology diffusion and economic growth; non-economic impacts of science and innovation expenditures; regional and global networks of knowledge generation and innovation; mechanisms for encouraging creativity and measuring outputs and outcomes from transformative research; and development, manipulation and visualization of data representing scientific activities. In addition to discussions and collaborations among the SciSIP researchers, an important benefit to the research community stems from discussions between researchers and leading practitioners. This network of scientists and practitioners constitutes a fruitful Petri dish that spawns exciting new scientific explorations in the SciSIP field.

Broader Impact: The conference facilitates interdisciplinary discourse between researchers in several fields. It also fosters communication and learning between academicians and policymakers. This activity advances the development of the SciSIP community of practice. The summary report is also a useful guide to data and tools that practitioners may utilize in day-to-day policy decision-making activities. The conference is also a rich venue for graduate students who are writing dissertations in this field.



22. Balancing the Portfolio: Efficiency and Productivity of Federal Biomedical R&D Funding

Researchers: Margaret Blume-Kohout, University of New Mexico and David Newman, University of California Irvine

Abstract: This cross-disciplinary, cross-institution collaborative research project combines economic analysis with state-of-the-art methods from statistical machine learning, to assess the relative efficiency and efficacy of research and development expenditures across the U.S. National Institutes of Health (NIH) portfolio of extramural projects.

Intellectual Merit: The novel combination of econometrics, topic modeling, and document classification permits analysis of massive collections of grant abstracts and scientific publications, identification of latent research topics present in NIH-funded research, assessment of possible spillover effects across research topics, and evaluation of causal linkages between changes in NIH funding by research topic and scientific advances.

Two specific research outcomes are considered: scientific publications, classified by topic; and pharmaceutical innovation, measured by drugs entering into clinical development to treat specific diseases. Planned research also includes refinement of existing economic theory to produce normative evaluations of the allocation of public research spending.

Broader Impact: This research will inform key policy questions related to federal funding of biomedical research. First, fiscal austerity requires careful attention to the nation's research portfolio and investments. This project will describe and evaluate the productivity of those investments, as a first step towards policy recommendations for rebalancing the portfolio, to maximize society's expected return on investment. Second, if NIH funding for basic research spurs increased pharmaceutical innovation, NIH possesses an important policy tool to promote pharmaceutical R&D in areas of high therapeutic importance or significant health disparity.



23. Effects of Immigrant Scientists and IPRS on Innovation

Researcher: Petra Moser , National Bureau of Economic Research

Co-Fund: Economics

Abstract: This CAREER research produces urgently needed empirical evidence on the effects of intellectual property rights and science policies on innovation. To investigate the effects of contemporary policy changes, which cannot be measured with contemporary data, it exploits substantial historical policy shifts that were exogenous to U.S. innovation. The research also constructs new data sets - including both patents and alternative measures of innovation - to improve our understanding of intellectual property rights and science policy on innovation in historical settings and today.

Intellectual Merit: Individual projects investigate the effects of immigrant scientists, compulsory licensing, patent pools, and the creation of patent rights for food crops on U.S. innovation. For example, the research uses the inflow of highly trained Jewish scientists, who fled the Nazi regime in Germany, as an exogenous event to help understand the potential benefits of receiving immigrant scientists. While existing studies of immigration have emphasized direct effects on wages and employment opportunities for native workers, this research investigates the potential long-run effects of receiving immigrant scientists on innovation by domestic scientists and engineers. Understanding these changes is essential because increases in innovation encourage economic growth and create new opportunities for employment. Complementary projects investigate effects of intellectual property rights policies, such as compulsory licensing, on innovation and examine improvements in food crops after the creation of intellectual property rights for plants in 1930 and in 1986 to test whether contemporary policies that strengthen intellectual property rights for biological innovations may help to secure world food supplies.

Broader Impact: Results from this CAREER research help guide government policies on immigration, patent laws, and innovation. Organizations like the USPTO and the World Intellectual Property Office (WIPO) have already requested presentations of new results on patent pools and plants patents. The CAREER-funded training of graduate students helps increase long-run research output on the impact of patents and innovative research-centered undergraduate courses encourage the dissemination of research output. Targeted mentoring and research collaborations increase the participation of minority students and women in academic research, and help to strengthen the representation of these groups among economists and scholars of innovation.



24. Estimating the Economic and Scientific Impact of Federal R&D Spending by Universities

Researchers: Jason Owen-Smith and Margaret Levenstein, University of Michigan Ann Arbor

Co-Fund: National Center for Science and Engineering Statistics (NCSES)

Abstract: This pilot project contributes to the scientific understanding of the role universities and federally funded research play in the larger economy. It does this by linking two rich and novel data sources -- the well- established and pivotal Longitudinal Household Employer Dynamics (LEHD) data program of the U.S. Census Bureau and the new STAR METRICS data being developed by the federal science-funding agencies. The goal is to estimate the impact of Federal academic R&D funding on regional and national economies. These data will be used to develop measures of the direct and indirect impact federal funding for academic science and engineering research has on human knowledge, technological developments, economic growth and resilience, and employment.

Intellectual Merit: The project characterizes different strategies for organizing university research and to use those differences to explain variations in the productivity of academic science. The data are also used to determine how research spending on campuses 'spills out' into the economy through subcontracts and vendor relationships, entrepreneurship, graduating students, and technological innovations. The ultimate goal is to document what campus expenditures mean for job creation, economic competitiveness and growth. Because this is a pilot study the project focuses on the social networks that underpin university research conducted in five states, and on economic spillovers in one state.

Broader Impact: The project develops technical tools that can be used for later, nationwide research will be developed that enable more effective use of existing data for the analysis of science policy and its effects. The analysis also informs efforts to expand or modify existing data collection in the STAR METRICS program, helping to insure that those data will be useful for policy makers and researchers alike. The work contributes to the development of an evidence based science of science policy and thus helps to develop tools to ensure that national investments in fundamental science and engineering research are effective.



25. 2012 Science and Technology Policy Gordon Research Seminar

Researchers: Susan Cozzens and Nancy Gray, Gordon Research Conferences

Co-Fund: Office of International Science and Engineering; Science, Technology & Society

Abstract: The Gordon Research Seminar on Science & Technology Policy is a unique forum for graduate students, post-docs, and other scientists with comparable levels of experience and education to present and exchange new data and cutting edge ideas. Junior scholars and practitioners in economics, science and engineering, science and technology policy, and science and technology studies will present their work.

Intellectual Merit: This year's GRS-STP is the first ever, organized by graduate students to accompany the successful series of Gordon Conferences on Science and Technology Policy; its theme is "The International Context of Science and Technology Policy". It will take place on the two days preceding the main GRC in Waterville, NH.

Broader Impact: The GRS is designed to strengthen the community of science and technology policy scholars, which it accomplishes in several ways. First, junior scholars in the several disciplinary areas of STS, economics, and science and technology policy have a chance to see what each of these perspectives has to offer the others and to make professional contacts that can help reinforce those connections as they develop professionally. Second, several senior figures in the field play mentor roles, including two economists and two policy scholars. The seminar includes a career panel of recent graduates who have gone into science and technology policy research or practice. Finally, this seminar represents a significantly higher level of diversity and inclusion than most Gordon Conferences, including multiple organizers and speakers who are African American and Latino and women, who are still under-represented in science and technology policy as a field. The organizers have reached out actively to HBCU campuses to maximize the participation of underrepresented groups and to build research capacity in the areas of science, technology, and society; economics; and the science of science and innovation policy.



26. TLS: Assessing the Impact of Science Policy on the Rate and Direction of Scientific Progress: Frontier Tools & Applications

Researchers: Jeffrey Furman, Scott Stern and Fiona Murray, National Bureau of Economic Research

Abstract: While the cumulative nature of knowledge is recognized as central to economic growth, the microeconomic and institutional foundations of cumulativeness are less understood. Although "Open Science" is widely recognized to play a central role in the production and diffusion of fundamental knowledge, few formal analyses support this understanding of the impact of policies and practices on the rate and direction of scientific progress.

This study focuses on the development and implementation of novel tools for quantitative analysis of the impact of science policy interventions on the process of cumulative scientific discovery. This analysis extends prior research by exploiting the recent availability of detailed citation data with frontier methods from the program evaluation literature. The approach moves beyond traditional cross-sectional comparisons of citations associated with knowledge in different institutional or policy environments; instead, "natural experiments" are utilized, where the conditions governing access, diffusion or follow-on research funding associated with a given piece of knowledge are changing over time. This approach allows the role of institutions and policy to be disentangled in shaping scientific progress from the intrinsic variation in scientific importance across discoveries. Specifically, the study outlines three types of tools: a differences-in-differences approach to citation analysis, the explicit comparison of changes in citation behavior in different subpopulations, and the development of an approach to recover the "distribution" of the impact of policy interventions. These tools can be fruitfully applied to provide novel policy analysis for a range of science policy interventions, from choices about the level of (and restrictions on) public funding, rules governing access to scientific research materials and data, and policies regarding intellectual property rights for discoveries resulting from the scientific process. In particular, the tools allow for the evaluation of science policy intervention on the rate and direction of scientific progress, and allows for the evaluation of the distributional consequences of policy initiatives. There are three potential applications of the tools in some detail, including (a) the impact of intellectual property rights on the diffusion and use of academic science, (b) the impact of national science policies on the geography and distribution of stem cell research, and (c) the impact of institutions that facilitate the sharing of research resources on the dynamics of knowledge accumulation in life sciences research. In sum, the outputs of the research will include papers that focus on the development of the tools per se, and papers that apply the tools in the context of applications focused on important science policy challenges.

This study will inform: (a) science policy analysis, by developing and implementing tools that can assess the impact of specific policies and institutions on the rate, direction, and composition of scientific activities; (b) the study of the economics of science, by elaborating the microeconomic and institutional foundations of knowledge accumulation, which supports economic growth; and (c) the sociology of science, by highlighting the roles of preexisting relationships, status, and networks in the expansion of the scientific community and explaining interactions between the features of the scientific system and its growth.

More generally, the broader impact of this study is to provide a set of tools for a range of science policy questions that have so far resisted quantitative analysis, and to offer a novel domain for the application and adaptation of frontier methods for program evaluation. In particular, the results of this research will have a specific impact in the economics and sociology of science, as well as in science policy analysis per se. In addition, the study provides a bridge between the explosion of quantitative data about science as a potential new area for the application of tools in the program evaluation literature.



27. Using Researcher Profiles to Demonstrate the Impact of Investments in Science

Researcher: Griffin Weber, Harvard University

Abstract: Evaluating the return on investment in science involves accurately associating research inputs (e.g., grants and contracts) with research outputs (e.g., publications and patents). Because many of today's significant discoveries are produced by large multi-institution, cross-disciplinary teams—often supported by different sponsors, with potential impact spread across different fields—accurately linking outputs to inputs can be challenging. This research investigates whether and to what extent an online profiling system that aggregates research data around the individual researcher facilitates the processes of linking research inputs to outputs and provides benefits to scientists, institutions, publishers, and agencies. It compares the value of using different data sources, such as federal systems, institutional repositories, commercial databases, manual data entry by librarians and administrators, and data entry by the scientists themselves, to populate a prototype website that profiles computer scientists from multiple institutions.

Intellectual Merit: The research breaks new ground in a variety of ways. It determines the cost and effort required to obtain each data source, including the resources needed to disambiguate names in order to link data on scientific contributions to the correct people responsible for them; it calculates the potential reduction in administrative burden each data source provides by measuring the amount of time it takes for a scientist, without the help of a researcher profiling system, to manually locate the data and enter it into an online form; and it evaluates which data sources contain the most information about high-impact cross-institutional or multi-disciplinary research, using data mining techniques to generate collaboration and topic-cluster maps.

This study is appropriate for the EAGER program because it is both high-risk—in that the outcome depends on the coordination of multiple software products, institutions, agencies and data sources in a rapid timeframe— and high-reward, in that a successful prototype would accelerate the implementation of a national researcher profiling system that benefits multiple stakeholders.

The field of Computer Science exemplifies all the challenges and potential rewards of a nation-wide researcher profiling system. Computer scientists are funded by many different agencies (NSF, NIH, DOE, DOD, NASA, etc.); their research outputs take many forms (publications, conference presentations, software, databases, algorithms, patents, etc.); and they collaborate across many disciplines (such as medicine, economics, engineering, physics, and social science).

Broader Impact: A publicly-accessible national researcher-profiling system based on linked open data promises numerous benefits beyond enabling more accurate measures of return on federal investment in science. These benefits include the potential to streamline the grant application and reporting process for researchers, identify reviewers without conflicts of interest, help researchers find collaborators, match trainees and junior investigators with mentors and jobs, and enable scientists to showcase their work.



28. Connecting Outcome Measures of Entrepreneurship, Technology, and Science (COMETS)

Researchers: Lynne Zucker and Michael Darby, National Bureau of Economic Research

Abstract: This project is completing the public COMETS database integrating data on science and technology inputs and outputs in order to better understand innovation and the success of high-technology firms.

COMETS and related on-line and on-site archives provide researchers the necessary tools to answer a wide range of important scientific and policy questions in the science of science and innovation policy (SciSIP). The project goes a long way toward eliminating the data bottleneck which has impeded SciSIP progress. It does this by integrating legacy databases on federal grants, universities, publications, dissertations, patents, and new and established high-technology firms in a single database with identifiers which link each appearance of an organization or scientist. The project maximizes the availability of fully public data at COMETS by developing and incorporating copyright-free and open-source alternatives to these data. Two demonstration projects (a) show the conceptual power and policy relevance of an integrated database combining public and licensed databases and (b) illustrate uses of COMETS while stress-testing the computer-matched codes used to identify each appearance of an organization or scientist.

Intellectual Merit: The intellectual merit of this project is based on conceptualizing and implementing construction of a very large coherent integrated database covering the national innovation system from government grants, laboratories, and policies through basic discoveries and their application to new commercial technologies driving the formation and transformation of high-technology industries. Reporting burden is minimized by using public information already collected for reference, reporting, marketing, and administrative purposes - unobtrusive measures that require no additional effort although a mechanism is provided for organizations and scientists to correct any errors in matching records within and across the legacy databases.

Broader Impact: This project has broad impact on SciSIP research and ultimately on economic growth and the standard of living. First, the project provides the research community with a platform technology which enables a quantum leap in the sophistication and reliability of SciSIP research since knowledge creation can be observed at the organization and individual scientist level. Second, since technological progress and rising educational levels are the two main factors determining growth in the advanced economies of the world, research enabled by this project can lead to improved science and innovation policies which increase technological progress and economic growth.



29. The Impact of Research and Development on Quality, Productivity, and Welfare

Researcher: Amil Petrin, University of Minnesota Twin Cities

Co-Fund: Economics

Abstract: Innovations in product quality and production methods are a major source of long run growth in output per capita. During this period of globalization it is very important to understand how investment in research and development (R&D) leads to innovations. Many countries subsidize activities that are intended to make firms more competitive in the world market. Empirical answers to questions regarding R&D require one to quantify how demand changes in response to the introduction of new goods and how supply changes in response to innovations in production.

Until recently almost all research on these questions used data that aggregate over prices and quantities so the measure of output is plant-level revenue. Aggregating prices and quantities over products at the plant confounds the effects of demand and supply, undermining the conclusions that can be drawn from such data.

More recent plant-level surveys include firm-level data on both the revenues and quantities of each product sold, in addition to data on inputs used in production and for product and process-related R&D. This research agenda will develop a methodology for estimating demand, output production and the production of innovations using these new and richer data sets.

Intellectual Merit: Applying the methodology will yield estimates that can be used to answer some of the long- standing questions related to R&D. A leading question in this literature is whether firms are doing too little or too much R&D from society's perspective, as economic theory says that forces can work in both directions.

Other important questions include how R&D "produces" innovations as a function of detailed input measurements, how spillovers of R&D knowledge impact growth, and whether past approaches to answering these questions with more aggregated data led to correct or mistaken inferences about the relationship between R&D and growth.

Broader Impacts: In an attempt to encourage firms to become more competitive in this age of globalization many countries including the United States subsidize R&D. By providing answers to the above questions this research will shed new light on the most effective way to allocate these subsidies. The data also come from a period when European Union tariffs were falling, making it possible to explore how stiffening competition affects R&D decisions, and how these decisions affect product and process innovations and the competitiveness of domestic firms. Our techniques will be programmed in Stata and made publicly available so researchers in other countries that collect similar data can directly apply the new methods.



30. Career Dynamics in the Science and Engineering Workforce

Researcher: Catherine Weinberger, University of California Santa Barbara

Abstract/Intellectual Merit: The current textbook explanation for gender differences in labor market outcomes among older workers, including (if not especially) scientists, is that women tend to fall behind men as they choose a less career-oriented work-family balance or as the effects of discrimination in promotion accumulate over the course of a career. Recent work finds the opposite to be the case when individual workers (or scientists) are followed over time. Women tend to earn less than men from a very young age, but tend to be on a similar, or even faster, growth path than men. The low average earnings of older women are predicted by their low earnings at labor market entry.

Intellectual Merit: The research examines whether Ph.D. scientists and engineers (either as a whole, or within subsets defined by field of study) tend to follow patterns similar to the highest-paid scientists at the bachelor's or master's degree level, as opposed to the overall patterns observed within samples of somewhat less educated workers, or whether new patterns of career dynamics are yet to be discovered. This study leads to new knowledge about the dynamics of career progress, the role of working long hours, and how these processes vary by gender and race.

Broader Impact: Benjamin Franklin made tremendous contributions to science and technology without devoting his entire life to science. Today, many scientists work long hours per week in an environment of intense competition. If particularly talented individuals with the potential to make enormous contributions to science are either discouraged from becoming scientists, or train to become scientists but do not receive the support that will allow them to do their best work, an opportunity is lost to all. Indeed, the 1981 Congress requested a periodic evaluation of the status of women and underrepresented minorities in the science and engineering workforce. The improved measurement and understanding of labor market processes and outcomes generated by this research should inform policies to efficiently promote gender equity, scientific productivity and the retention of talented scientists within a population of workers in whom we as a nation have invested enormous resources.



31. Investing in Science, Research, and Technology: Where is the Biggest Band for the Buck?


Researchers: Sandy Dall’erba and Jaewon Lim, University of Arizona

Abstract: As regional economies increasingly compete at the national and international levels to develop their innovative edge, helping regions identify and support the factors at the origin of innovation has become one of the most important tasks of regional economic developers. This project is based upon the knowledge production function literature which relies on a statistical approach to estimate the impact of investments in R&D (Research and Development) on the production of innovation. Because this project focuses on the production of innovation at the regional level, it requires us explicitly to account for the role of knowledge externalities between regions, a phenomenon brought to the fore by the new literature in economic geography. As of today, however, only a handful of studies use the appropriate statistical techniques to model and measure interregional knowledge externalities; when they do, they make global (not local) measurements which mask significant variations in the characteristics of innovation across regions.

Intellectual Merit: This work improves upon the traditional approach by providing measurements of knowledge spillovers at the local level, i.e. for every single region. This work also investigates the dynamics of regional knowledge creation for four "high-technology" sectors to better substantiate differences in knowledge externalities across sectors. In addition, the statistical analysis is performed on three different spatial scales (US counties, Metropolitan Statistical Areas and States), making this work more complete and informative to local governments than traditional approaches. Another contribution is a method of measuring the amount of income and jobs created by R&D investments, i.e. their social returns. This contribution is accounted for through the modeling of interregional externalities by means of two complementary techniques called spatial econometrics and multi-regional input-output. Last but not least, the dataset used in this project significantly improves upon previous studies. Not only do the investigators rely on the most recent private R&D investment data from the US Patent and Trademark Office, but they also use the Consolidated Federal Funds Report of the Census Bureau to estimate, for the first time, the role of federally funded R&D investments on knowledge creation at the regional level. Overall, this project aims at showing where to maximize R&D investments and help regional governments tailor appropriate policies to promote knowledge creation.

Broader Impact: The broader impacts of this project are threefold: first, the thorough statistical approach used in this project will help local policymakers identify the key economic characteristics they should rely on to stimulate knowledge creation in their locality. Second, a clear estimation of the relative returns on public investments is essential to support evidence-based fiscal policy. The private and social returns on R&D investments evaluated in this work can thus be directly compared with the returns from other public investments such as education, transportation and defense, for instance. Finally, in addition to the usual scholarly publications and in-class presentations, the results of this project will be accessible online and for free in the frame of a dynamic WebGIS (Geographic Information System). Because it is able to give a large audience access to all the results, this dissemination method promotes transparency, objectivity and accountability in the research at hand.



32. EAGER: Unified categories for describing and quantifying scientific research


Researcher: David Newman, University of California-Irvine

Abstract: This project develops state-of-the-art machine learning methods to describe and quantify scientific research. The particular approach taken is to develop new topic models that can learn underlying research categories across a wide variety of text data sources including NSF and NIH grant awards, scientific publications, and US patents. The important innovation is the building of the technology to permit feedback from domain experts and end users. The approach is potentially transformative in that it can potentially overcomes some of the known limitations of current topic modeling approaches by improving the quality and utility of topics across diverse data sources.

The web-based tool displays and manipulates learned topics so that users can apply it to create comprehensive overviews of scientific funding, research, and production, including the answers to questions like:
- What types of science are funded by NSF, NIH and other agencies?
- What types of science are produced by funded investigators?
- What types of science are described in US patents?
The tool also provide users with answers to more complex questions about the science of science and the relationship between funding and scientific achievements, tracking trends, describing funding programs, and identifying funding overlap (across agencies, or even within agencies).

Intellectual Merit: The proposed research advances techniques and methods for tracking scientific research in several ways. First, it advances the development of unsupervised statistical topic models to categorize, describe, and measure scientific research. Second, it addresses known problems with topic modeling for this type of application, such as improving the coherence of all topics, and making topics transcend different types of document collections (grants, publications, patents). Users can more directly measure impacts of funding as a result of being able to make use of unified topics from grants, publications, and patents. Third, the research develops evaluation frameworks that shift the focus from machine learning metrics to the needs of domain experts and end users.

Broader Impact: This work has an array of broader impacts. It creates useful data for funding agency staff, researchers, interested public, government bodies, media and other stakeholders. The web based tool allows users to create custom-based data sets, tailored to their particular needs. Such data sets allow users to answer an array of science of science policy questions. The knowledge created in this work supports initiatives such as STAR METRICS to document the value of investments in scientific research.