A. Understanding science and innovation:

1. A National Survey of Organizations to Study Globalization, Innovation and Employment. (Clair Brown, University of California-Berkeley)

2. Tom Edison and the Electric Innovation Machine. (Gary Bradshaw, Mississippi State University)

3. Science & Technology Innovation Concept Knowledge-base (STICK): Monitoring, Understanding, and Advancing the (R)Evolution of Science & Technology Innovations. (Ping Wang, Yan Qu and Ben Shneiderman, University of Maryland)

Modeling Innovation:

4. A Predictive Simulation Model of Competitive Dynamics in Innovation. (Risto Miikkulainen, University of Texas, Riitta Katila, Stanford University)

5. Modeling Schumpeter's Theory of Innovation as a Basis for Innovation Policy: An Experimental Approach. (John Gero, George Mason University)

6. Co-Evolution of Innovative Products by Purposive Agents and the Growth of Technological Complexity. (Robert Axtell and William Kennedy, George Mason University)

7. Firm Innovation, Selection and Labor Market Frictions. (Rasmus Lentz, University of Wisconsin Madison)

Tracking science and innovation:

8. Tracking Scientific Innovation from Usage Data: Models and Tools to Support a Science of Science. (Johan Bollen, Los Alamos National Lab, Carl Bergstrom, University of Washington)

9. Assessing and Predicting Scientific Progress through Computational Language Understanding. (James Evans, Ian Foster and Andrey Rzhetsky, University of Chicago)

Scientific networks and science outcomes:

10. Mapping the International Evolution of Collaboration Networks on Patents Granted to Universities around the World. (Margaret Clements, Indiana University.)

11. The Influence of Network Structure on Sex Disparities in Scientific Collaboration: Commercial Innovation in the Life Sciences. (Kjersten Whittington, Reed College)

Science and innovation policy:

12. Compulsory Licensing - Evidence from the ’Trading with the Enemy Act.”(Petra Moser, Stanford University)

13. Metrics for Capturing Crucial Social Dynamics of Innovative Regions: Implications for S&T Policy. (Mary Walshok, University of California San Diego)

14. Where are all the Female Engineers? (Jeffery Smith, University of Michigan, Dan Black and Robert Michael, University of Chicago)

15. An Experimental Producer Price Index for Clinical Trials. (Ernst Berndt, National Bureau of Economic Research and Ian Cockburn, Boston University)

16. Human Capital and Career Mobility in Science and Engineering-Intensive Start-ups: An Open Access Initial Public Offerings Database. (Martin Kenney, University of California Davis)

17. Scientists and Engineers as Agents of Technological Progress: Measuring the Returns to R&D and the Economic Impact of Science & Engineering Workers. (Richard Freeman, Erling Barth and Andrew Wang, National Bureau of Economic Research and Gerald Marschke, Harvard University)

Describing innovation:

18. Applied Visual Analytics for Economic Decision-Making. (David Ebert, Purdue University, Timothy Cason, Purdue University, David Hummels, Purdue University, Anya Savikhin, Purdue University)

19. A Visual Analytics Approach to Science and Innovation Policy. (Martin Ribarksy, Remco Chang and Jim Yang, University of North Carolina at Charlotte)



1. A National Survey of Organizations to Study Globalization, Innovation and Employment

Researcher: Clair Brown, University of California-Berkeley (cofund with IOS and Sociology)

Abstract: At this time, the U.S. has no data resources that permit satisfactory measurement and analysis of three key processes in our economy -- globalization, technological change, and innovation. Better measurement of the effects of these forces could enable better decision making in many areas pertinent to competitiveness and economic well being.

Intellectual Merit: This project makes an important contribution by collecting and analyzing new data on global engagement, use of technology, and innovation-related activity by United States organizations.

The project involves conduct of a National Organization Survey (NOS), which will link to items in the well-regarded General Social Survey (GSS), and which will be made available for broad dissemination to the research community. The data will be of use to policy analysts as well as scholars in organization sciences, economics, and geography, among other fields, enabling them to address many important topics pertaining to both scientific and policy debates.

The survey uses a dual frame approach. The first frame consists of a nationally representative sample of public sector and other not-for-profit organizations as well as for-profit firms derived from and linked to a nationally representative survey of workers, the General Social Survey (GSS). The second frame consists of a representative sample of large for-profit firms, which are those that are most likely to be globally engaged innovators. The dual frame survey will allow the team to study how an organization’s domestic jobs relate to its actions regarding innovation, use of technology, outsourcing, and off-shoring.

Broader Impact: This is the first National Organizational Survey to collect significant data on activities by business functions. It is also the first time that data on business function outsourcing and off-shoring has been collected from a representative sample of United States organizations. With this data, analysis can go beyond recent globalization studies that estimate the number of American jobs that are potentially off-shorable, and begin to systematically examine what firms and other organizations are actually doing in regard to both outsourcing and off-shoring.



2. Tom Edison and the Electric Innovation Machine

Researcher : Gary Bradshaw, Mississippi State University.

Abstract: Although technological innovation is seen as essential to the success of the US economy, our discussions about the nature of innovation are all too often fuzzy, relying on vague notions of genius, luck, perseverance, or Yankee ingenuity. However, if government or private business are to make strategic investments in technology, then it is important to develop a better understanding about how individuals create new technology. This research examines discovery and invention as cognitive processes to advance understanding of technological creativity. The particular focus is on one of the most prolific U.S. inventors, Thomas Alva Edison. His inventions include the kinetoscope/kinetograph, which led to the motion picture industry); the phonograph, which led to the recording industry; and the electric light, which led to the electric generator and power grid. In his day, few questioned the processes that made Edison so successful: He was regarded as a genius beyond the reckoning of ordinary people. Even today, our knowledge of Edison is a complex mixture of myth and fact. Thus the nature of his success remains a considerable mystery: How did Edison produce such a remarkable string of successes? By studying his inventive processes in detail, this research advances the understanding of the psychology of invention and innovation, in particular the thought processes that help inventors produce practical inventions that are commercially successful.

Intellectual Merit: This project analyzes Edison's notebooks, correspondence, and artifacts to reveal the methods Edison used as an inventor and innovator. The focus of the research is understand invention as the movement of ideas across three spaces: experimental, design, and conceptual. This research reveals the mental models Edison used to think about his inventions, the heuristics he used to create and improve his inventions, and the strategies Edison pursued in coordinating a complex set of activities.

Broader Impact: This research identifies creative methods that can be taught to engineers and innovators in order to help them to produce innovative products. Managers, as a result of understanding the creative processes that underlie innovation, should be able to evaluate whether the prerequisites for innovation are met and to track innovative progress (or lack thereof). Finally, this research makes fundamental contributions to the psychology of creativity and invention.



3. Science & Technology Innovation Concept Knowledge-base (STICK): Monitoring, Understanding, and Advancing the (R)Evolution of Science & Technology Innovations

Researchers : Ping Wang, Yan Qu, Ben Shneiderman, University of Maryland.

Abstract: This project provides much needed data and tools for analyzing innovations of all possible outcomes, included failed innovations. This approach overcomes the bias in the science policy which studies only popular or ultimately successful innovations.

Intellectual Merit : This comprehensive endeavor enables SciSIP researchers to build and test theories that explain the differentiated trajectories of science and technology innovations and their associated communities. The project also spans disciplinary boundaries by bridging the artificial divide in SciSIP research between the production and the use of innovations, piecing together a holistic view of the dynamic supply and demand in the innovation ecosystem. Specifically, the project builds a large-scale, multi-source, longitudinal database, Science & Technology Innovation Concept Knowledge-base (STICK), and develops a set of visual analytic tools for monitoring and understanding the emergence and revolution/evolution of innovations in three exemplar science and technology fields: information technology, biotechnology, and nanotechnology.

The knowledge-base captures innovations, the individual and organizational actors associated with the innovations, and the relationships among the innovations and the actors through a hybrid approach that combines computational analysis of text (e.g., natural language processing) and social information processing (e.g., social tagging and collaborative writing by the users of the knowledge-base). State-of-the-art visualization tools are customized for SciSIP researchers and other innovation stakeholders to visualize innovation networks and analyze patterns and trends. The design of the knowledge-base and toolset is grounded in a demonstration study on the popularity of innovations. The study aims to address important questions concerning the complex relationships among innovations and the evolution of communities, with implications to the popularity and ultimate success of innovations.

Broader Impacts : STICK is institutionalized at the University of Maryland at College Park as a free public service that offers web access to the data and tools developed in this project. This service also produces quarterly reports on the status of science and technology innovations, including the National Innovation Popularity Index, analogous to the Consumer Confidence Index for the state of the economy. This research-based service is an intuitive tool for science and technology education. For most fields where specialization is the theme, students' and the public's interests increase with the capability to monitor and make sense of the fast-changing arenas where innovations emerge, converge, and diverge. For scientists and engineers, STICK's visual analytic toolset helps accelerate scientific discoveries and innovations by identifying and establishing collaborations within and across innovation communities. Finally, STICK helps science and technology policy makers monitor and understand the evolutionary paths of innovations, appraise the significance of innovations in rigorously charted terrains, and proactively foster, promote, and advance innovations with benefits to the society.



4. A Predictive Simulation Model of Competitive Dynamics in Innovation.

Researchers : Risto Miikkulainen, University of Texas, Riitta Katila, Stanford University (cofund with IOS)

Abstract : How does competition influence innovation? This project takes a unique approach to understand this question: It develops an evidence-based simulation platform to understand how competitive dynamics influences innovation. This approach contributes new knowledge to technology strategy and a transformative new tool for science and technology policy.

Intellectual Merit: There are three main contributions. First, a computational multi- agent search theory is developed and evaluated for environments with complex interactions of competing agents. The results help understand search for innovation when multiple agents search simultaneously, and develop computational representations of it. Second, qualitative observations on innovation and competition are made through structured interviews in a real technology-based industry. The goal is to provide theoretically informed answers to an enduring question in technology strategy: What are the most effective firm-level innovation search strategies in competitive markets? Third, a software research tool called ASaP is developed by integrating insights from the computational and strategy components of this project.

This simulation tool makes it possible to perform "live" simulations of industry data, similar to physical simulations in engineering. Scholars and public policy makers can use this tool to understand the ways in which their recommendations are likely to play out and therefore help design effective policies to manage competition and simultaneously promote innovation.

Broader Impacts: The results of the project can be used not only by academics to analyze competition and innovation rigorously but also by business and policy analysts to improve innovativeness of firms and industries, and thus ultimately advance economic productivity. To foster such progress, the ASaP tool is made publicly available to scholars and practitioners through a website, thus lowering the barrier of entry to computational analysis in general, and predictive analysis of business data in particular. During the current difficult economic times, it is more important than ever to remain innovative, both to resist deeper downturn and to bring our technology-based economy back to a growth trajectory. The ASaP tool serves as a live model of the data, and thus can be used to identify the sequence of events that generated the data, and modified to find out what would have happened if some factors (such as supporting the livelihood of certain players in the industry over others) had been different. In essence, ASaP makes it possible to study the archival data interactively in laboratory-like experiments, and it can therefore lead to insights on science and technology policy that are not possible to obtain otherwise.



5. Modeling Schumpeter's Theory of Innovation as a Basis for Innovation Policy: An Experimental Approach

Researcher : John Gero, George Mason University

Any science requires the following elements: phenomena; empirical data that describe those phenomena; hypotheses at various levels to connect, explain and then predict those phenomena; and the ability to test those hypotheses, Once a hypothesis is able to predict it becomes a theory. The science of innovation policy must be based on a platform that permits the formulation and testing of key hypotheses. Constructing such a platform is typically a challenge in the social sciences as there is often no ability to carry out experiments in the same way that the physical sciences can, i.e, in a laboratory. This project develops an approach to test hypotheses and theories by creating a computational laboratory within with hypothesis evolution and hypothesis testing can take place.

Intellectual Merit : Much of the current research in the science of innovation policy is focused on collecting empirical data. There is a surprising paucity of testable models and theories. There are two directions from which models can come: empirical observation or inference based on a qualitative understanding of a field. Schumpeter, in his analysis of the success of capitalism, has produced one of the most explanatory qualitative theories of innovation known as ?creative destruction?. This theory of innovation takes a Lamarckian view of the evolution of products and processes, but it has yet to be formally modeled and tested. The contribution of this research is to advance the modeling and testing of Schumpeterian creative destruction. In doing so the project creates a laboratory, which can then be used more generally. The laboratory is designed to synthesize concepts from the disparate fields of: innovation theory; creative production; computational sociology; social multi-agent systems; situated cognition; emergence; and data mining. Both the laboratory and the results it produces provide the foundations for a science of innovation policy: science that produces testable results, and one that can test hypotheses. The laboratory uses computational sociology, a technology based on social multi-agent systems that allow for emergent behavior, as the modeling tool. Agents are generators and receivers of “products” and take up novel, useful and unexpected products. The overall system behavior is structured to be emergent and is captured using data mining techniques. Because the system connects inputs to outputs at the overall system level, the effects of different types of innovation policies can then be tested in the laboratory.

Broader Impact : The broader impacts of this research lie in multiple dimensions. The project will involve PhD students and give other students experience with this kind of integrative research. It will make connections to computer science, cognitive science, social science and design science. The results from this project provide as feedback to design and innovation educators initially at George Mason University and then to design and innovation educators at other universities through the use of demonstrations. The results from this project are disseminated via conference papers, journal papers and a website. The laboratory is publicly available publicly so that others can experiment with it through the website.



6. Co-Evolution of Innovative Products by Purposive Agents and the Growth of Technological Complexity

Researchers : Robert Axtell, William Kennedy, George Mason University.

Abstract: Innovation is a driving force in the economy, and is often accompanied by technological complexity. Innovation can naturally be modeled as the co-evolution of a variety of economic goods accomplished by purposive individuals. This project draws on the research literature associated with biological evolution together with more conventional economic models of technological innovation in order to create an agent- based computational model in which purposive actors (simulated as agents) invent new projects, new products diffuse into the economy, the number and diversity of economic goods increases over time, and the technological complexity of the economy grows alongside the agents’ welfare.

Intellectual Merit : This class of models is capable of generating many of the stylized features of actual innovation systems, such as perpetual change, creative destruction, continuously increasing human living standards, and complex chains of inter-related technologies. By creating a high-fidelity model of the technological innovation process, a better understanding of how policy catalyzes innovation is developed. This is a fertile time to make cross-disciplinary approaches to models of innovation, by mixing state-of- the-art computer science techniques with what is known today about innovation within the social sciences. This project includes elements from evolutionary computation, social network analysis, and behavioral economics, and promises to be a new way to investigate the science of science policy.

Broader Impacts: Innovation affects all aspects of modern life, and broader impacts are implicitly part of any improved understanding of scientific discovery and industrial commercialization processes. Specifically, the project exposes the kinds of models built by this research to a variety of audiences outside the basic research community. The outstanding Thomas Jefferson High School for Science and Technology, near George Mason University, is the location of one such outreach activity. There, seniors in computer science classes may elect to execute a year-long project on a topic of their choice. Increasingly, they choose topics in the social sciences and build agent-based models to realize them. The researchers on this project work with computer science teachers there to engage some of their students to work on models related to science and technology policy.



7. Firm Innovation, Selection and Labor Market Frictions. (Rasmus Lentz, University of Wisconsin Madison)

Researcher : Rasmus Lentz, University of Wisconsin Madison.

Abstract : The project adopts the view that firms differ fundamentally in the extent to which their innovations contribute to aggregate productivity. Ideally, the most creative and innovative firms should expand their operations at the expense of the less creative firms, which would increase aggregate productivity. However, firm panel data document the co-existence of firms with very different productivity levels, suggesting that the selection process is imperfect.

Intellectual Merit: There is very little empirical work done on firm innovation based on micro panel data. The project breaks new ground in making inferences about innovation processes based on firm panel and worker flows data.

The project studies the impediments to the selection process. One impediment is the innovation process itself limits expansion by requiring costly investment in R&D with an uncertain outcome. A second impediment is due to the firm's costly acquisition of labor inputs from the labor market. The expansion of activities by more creative firms requires worker reallocation from less to more creative firms. If the labor market fails to facilitate such reallocation, returns to innovation are reduced.

The project studies a broad range of policies that affect the selection process. This includes the study of research and development subsidies and patent law design, but also importantly the impact of labor market policies on innovation. Policies such as mandatory severance payments, minimum wages, unemployment benefit design could potentially affect innovation in non-trivial ways to the degree that it reduces the market's ability to reallocate workers between firms.

The empirical part of the project is based on U.S. and Danish micro panel data.

Broader Impacts: The project informs policy decisions in the areas of research and development subsidies and patent laws as well as a long range of labor market policies in relation to their impact on firm innovation and aggregate productivity. Furthermore, the project produces a coherent view of the structure that governs firm innovation and growth. The project relates this structure to modern micro panel data. These types of data are typically confidential and subject to restricted access. The project sets forth estimation methods that facilitate research replication and transparency.



8. Tracking Scientific Innovation from Usage Data: Models and Tools to Support a Science of Science.

Researchers- Johan Bollen, Indiana University, Carl Bergstrom, University of Washington.

Abstract: This project develops a set of tools that allow organizations investing in Science and Engineering to identify and predict the emergence of innovative research. Such a capacity would permit organizations to efficiently allocate resources to stimulate rapid and effective research process in these areas. Several key attributes are needed: the tool should be able to operate in real-time, be representative of the widest possible sample of scientific activity, and support a cost-benefit analysis of allocated resources.

Intellectual merit : This research aims to support the development of such tools by focusing on two scientific and methodological issues. First, the project studies the potential of early indicators of scientific activity such as usage data and search query logs. Second, the project aims to develop models that can, on the basis of such early indicators, identify and predict emerging trends in real-time. The project leverages the efforts of two well-established projects, namely the MESUR project ( and the Eigenfactor project ( The MESUR project has, over the course of the past 2 years, captured a significant sample of the world?s scientific activity, via a collection of more than 1 billion article-level usage events acquired from some of the world's most significant publishers, aggregators and university consortia. The Eigenfactor project has demonstrated the power of mathematical network models (cf. Google's PageRank) to rank disciplines and journals according to the lattice work of scientific citations that records the collective history of S&E research. Predictions of the "flow" of scientific activity have been used to produce detailed maps of scientific activity that may identify potential foci of scientific innovation. This project expands the Eigenfactor models to include MESUR's indicators of actual, real-time scientific activity. On that basis the project develops a set of early indicators that can detect the emergence of scientific innovation in real-time - before such trends are visible in citation data - and relates these indicators to public policy and decision making. The project also develops explanatory and predictive frameworks that connect observations of individual behavior with emergent, collective phenomena such as scientific innovation. Since the focus of the research is whether it is possible to develop analytic and predictive tools that indicate why, how and where scientific innovation is most likely to occur, the existing services will be leveraged to produce freely available, expandable tools that rank, analyze, predict and chart areas of scientific innovation.

Broader Impact: this research project produces freely available, expandable services to form an "early warning" system for scientific innovation that are expected to lead to a better public understanding of science as a complex, dynamic system. Such services should foster public participation in efforts to establish a more diverse, innovative research landscape that can meet the challenges of the 21st century.



9. Assessing and Predicting Scientific Progress through Computational Language Understanding.

Researchers : James Evans, University of Chicago, Ian Foster, Argonne National Laboratory, Andrey Rzhetsky, University of Chicago. (cofund with Chemistry)

Abstract: This project provides new approaches to the evaluation of scientific and technological promise. The basic insight is that scientific concepts, like organisms within ecologies, only exist in networks of supporting ideas, and this is key to understanding the way in which scientific concepts are adopted and diffused. The particular discipline that is studied is chemistry and related disciplines. The work has three stages. First, it assesses and predicts innovation in science from the novelty and popularity of terms and statements within a scientific network.

Second, it assesses and predicts the integration of scientific knowledge from term and statement linkage, repetition, and elaboration patterns. Finally, it assesses and predicts success along the path from science to technology by linking term and statement connections to problems. The research uses cybertools to develop a very large database of scientific terms and statements across a broad corpus of published research and invention, including news, blogs, and other informal text as well as unpublished opinions.

Intellectual Merit: Scientific evaluation, from awarding grants to reviewing tenure, has historically relied on quantity to proxy for quality. Progress is inferred from the amount of research produced or the sum of attention garnered. Numbers of books, articles, pages, citations and media mentions are tallied. These quantities are inexpensive to measure, but fail to directly capture whether a contribution is important. This project advances the measurement of scientific achievement by placing scientific claims in the context of past science. It does this by building on recent advances in computational language understanding, and the electronic availability of science. In particular, the project extracts scientific term and statements from a broad collection of published articles, patents and blogs in disciplines related to chemistry.

These statements are supplemented with information about their social context -- their location in the network of authors and the geographical sprawl of global research institutions. Models are then developed that exploit patterns in the structure of scientific language to assess the importance of scientific programs and fields. The degree of innovation in science is assessed from the novelty and popularity of terms and statements within the broader network. The integration of scientific knowledge is assessed by examining the term and statement linkage, as well as repetition and elaboration patterns. These are, in turn, used to predict the path from science to Technology. The project also develops new methods for managing and processing large quantities of text and network data.

Broader Impacts: The project develops general methods relevant for policy makers and scientists. This research generates, for example, high resolution, dynamic maps of knowledge claims in chemistry and neighboring disciplines such as pharmaceuticals and toxicology. The interactive nature of the maps means that they can serve as a teaching tool to help students understand scientific trends in their corner of science. They can also facilitate precise analysis of the production of science and stimulate the production of new hypotheses, as researchers note statements not made within the network of claims. When these maps are combined with the scientific models, they hold the potential to revolutionize the way scientists collaborate, identify research problems and validate hypotheses. Finally, because the research both clarifies what is published and where, as well as traces the careers of scientists and inventors, the research generates insights into what factors channel scientific attention, and how these factors can be harnessed to guide the most powerful public investments in innovation.



10. Mapping the International Evolution of Collaboration Networks on Patents Granted to Universities around the World.

Researcher : Margaret Clements, Indiana University (cofund with STS).

Abstract: This project studies how knowledge flows between and among academic scientific inventors around the world. University patents are the object of analysis for this study because patents reflect both knowledge production and diffusion. By analyzing collaborations between inventors and institutions on patents granted to universities around the world, the following questions are addressed: How does knowledge flow between academic inventors? How do country characteristics, national policies, and institutional practices shape knowledge diffusion? To what extent do individual academic inventors direct the flow of knowledge?

Intellectual Merit: From a policy perspective, the international impact of the Bayh-Dole Act--a bipartisan legislative effort to facilitate technology development and transfers between industry, the academy and federal laboratories--is examined. This study traces knowledge diffusion as a complex system by combining methodological approaches that include network analysis, network visualization, patent citation analysis, temporal analysis, and topical analysis. This project: demonstrates important relationships between academic inventors and inter-organizational networks around the world; analyzes patents as an artifact of important processes of knowledge diffusion; and describes the shape, structure and evolution of scientific collaborations on university patents around the globe.

Broader Impact: The findings of this study will provide substantive information that may be of use to organizations and science and technology policymakers in designing and implementing policies for knowledge diffusion and innovation. This study intends to make fundamental contributions to theoretical knowledge about complex systems and processes of knowledge diffusion.



11. The Influence of Network Structure on Sex Disparities in Scientific Collaboration: Commercial Innovation in the Life Sciences.

Researcher: Kjersten Whittington, Reed College.

Abstract: This research examines the influence of network structure on sex disparities in scientific collaboration. It incorporates data on the collaboration networks of the national population of biotechnology inventors and those from a variety of organizational settings (including public research organizations, pharmaceutical companies, and government laboratories) across a thirty-year period (1976-2002). The results should provide direct causal linkages between network mechanisms and gendered outcomes in science.

Intellectual Merit: Previous research on sex differences in social connectivity suggests that women and men assume qualitatively different patterns of interaction in their work settings. These network theories of influence on women's performance suggest that a key mechanism in the production of sex disparities manifests itself through the ties and connections that women make, or are able to make, in workplace settings. Little is known about the ways in which network mechanisms might be operating to produce differences in outcomes across scientific settings, however. It is not clear what types of networks support women's collaboration and whether those factors differ from those that support men's collaboration. And while it has been shown that sex disparities in scientific productivity appear to differ across organizational settings -- specifically, in more horizontally-organized work settings as compared with more hierarchical ones.

The mechanism for these differences has not been clearly identified. Greater insight into the stratification processes of women and men in science can be gained by studying how organizational forms affect the way collaborative work is structured across sectors. The premise of the proposed study is that the differing structures of collaboration in particular organizations or industries not only influences men and women scientists’ opportunities for finding collaborators, but the subsequent influence of these connections on productive work more generally. The transformative nature of this research lies in the integration of network theories of social structure and current insight in work on gender and occupational dynamics within a national innovation context. In addition, this research employs novel research methods that extend our current approaches to network analysis.

Broader Impact: This project has significant implications for current discussions on the causal factors bringing about sex disparities in science and innovation, and also for the larger theoretical literature on networks and social structure. This research establishes whether network mechanisms across organizational forms result in decreased innovation by women. As such, it provides insights into how policy can be targeted to fully engage women in science and contribute fully to national scientific innovation.



12. Compulsory Licensing - Evidence from the ’Trading with the Enemy Act.”

Researcher: Petra Moser, Stanford University (cofund with Economics).

Abstract: Compulsory licensing has been advanced as a policy tool to deliver life-saving drugs to millions of patients in developing countries and in the United States. Under this policy, which is permissible under the Trade Related Intellectual Property Rights (TRIPS) agreement, governments grant domestic firms the right to produce inventions that are patented by foreign nationals, without the consent of patent owners.

Compulsory licensing offers obvious short-term benefits as it grants quick access to medicines and other essential innovations. The policy’s long run effects, however, are unclear. On the one hand, compulsory licensing may reduce incentives to invest in R&D as it weakens the property rights of original inventors. On the other hand, compulsory licensing may promote invention as it enables a new set of firms to gain experience with production, which in turn creates opportunities for learning by doing and strengthens incentives to invest in scientific training and other skills that are necessary for invention.

Intellectual Merit: This project uses an exogenous event of compulsory licensing after World War I to measure the long-run effects of compulsory licensing on domestic invention in the licensing country. Specifically, we compare changes in patents by domestic inventors across U.S. chemical inventions that were differentially affected by compulsory licensing under the Trading with the Enemy Act (TWEA) of World War I.

Broader Impact: Preliminary tests on a subset of chemical patents suggest that compulsory licensing has a large positive effect on domestic invention. Preliminary tests also suggest that the full effects of compulsory licensing set in only after about 10 years, even though some effects appear after about 5 years. Thus, due to their long-run nature, the full effects of compulsory licensing may be missed in analysis of contemporary data.



13. Metrics for Capturing Crucial Social Dynamics of Innovative Regions: Implications for S&T Policy.

Researcher: Mary Walshok, University of California San Diego.

Abstract: This research quantifies and documents the role of social dynamics in the economic growth and development of regions with great scientific institutions. The presence of major scientific research institutions in a region does not, by itself, lead to social and economic benefit in that region. The importance of hard assets such as R&D expenditures, patents and venture capital, as well as the critical mass and propinquity of scientific talent is well known. However, this research probes in detail a set of social mechanisms which are postulated to enable some regions to transfer and commercialize knowledge more effectively than others. It examines the importance of the robust activity of S&T boundary-spanning organizations in a region in contributing to successful technology commercialization as measured by number of new business startups. These S&T boundary-spanning organizations, social gatherings of cross- disciplinary and cross-functional groups including both technologists and their business supporters, are of three types: formal groups that support local technologies (trade associations and entrepreneur-aiding groups); volunteer-run chapters affiliated with national organizations (e.g. IEEE, AeA, Sigma Xi, AWIS); and informal volunteer-run affinity groups focused often on a particular interdisciplinary subject such as systems biology or nanotechnology.

Intellectual Merit: Previous research on social dynamics, largely descriptive and anecdotal, has yet to inform policy. This project studies and quantifies the following aspects of the region's boundary-spanning groups: their types, numbers, meeting frequency and attendance, their membership diversity (Shannon-Wiener index), and the region's overall membership overlap among these boundary-spanning groups. These independent variables are then correlated with new business startups including controls for R&D expenditures, patent activity, venture capital, critical mass and propinquity of the region’s scientific workforce.

An initial set of three pilot case studies represent the first step in the development of a predictive model based on machine learning techniques. A recommended data structure is being developed so that other research institutions or their regions may also gather data to perform self assessments and thus add to the training and test data used to test and enhance the resulting model.

Broader impacts: This work develops metrics which capture "cultural and social dynamics" and correlates them with successful knowledge flows between research institutions and a dynamic regional commercialization ecology. It searches for the social factors that distinguish vibrant regions with the capacity to innovate from those that do less, in order to better understand the processes by which investments in S&T research are transformed into positive social and economic outcomes.

This information may influence how regions and research organizations set policy to stimulate social networks and inclusive interdisciplinary discourse groups. The increased understanding of previously-uncharacterized aspects of knowledge flows should lead directly to new mechanisms available for optimization by policy makers. The application of artificial intelligence machine learning techniques in this work should also lead to new capabilities in the science of science policy.



14. Where are all the Female Engineers?

Researchers: Jeffrey Smith, University of Michigan Ann Arbor, Dan Black, National Opinion Research Center, Robert Michael, University of Chicago

Abstract: Perhaps no issue is more important for US science policy than understanding why qualified women (and men) do or do not choose careers in science and engineering. The quality of a nation's scientific workforce determines, in large part, the quality of its science. If qualified women avoid careers in science and engineering, science and the nation suffer.

Since World War Two, women have been increasingly attracted to technical fields. Women have recently become a majority of college students and college graduates, they represent a majority of applicants to medical school, a majority in accounting and auditing, and they apply to law programs in numbers equivalent to men. In addition, women have been gaining in performance on quantitative skills tests. In 1979, women had a 2 percentile point deficit relative to men on the mathematical ability test on the National Longitudinal Survey of Youth; but in the 1997 cohort they had a 4 percentile point advantage. Yet women constitute less than a third of degree holders in the physical sciences and hold only ten percent of recent degrees in engineering.

Intellectual Merit: This project seeks to understand why women have flooded into some traditionally male fields, but not into others, despite their increasingly strong technical qualifications. The research team addresses this question via analyses of the effects of starting careers in science and engineering on marriage rates, divorce rates and fertility rates. They will also consider how labor market interruptions affect the lifecycle path of women's earnings in various fields.

Broader Impact: These analyses directly address concerns in the literature that science and engineering impose relatively (compared to other fields) large costs on family formation. The researchers will compare the results from US data with results from parallel analyses of British data conducted by collaborators in the UK.



15. An Experimental Producer Price Index for Clinical Trials.

Researchers: Ernst Berndt, National Bureau of Economic Research and Ian Cockburn, Boston University (cofund with Economics and BEA)

Abstract: This research project studies the feasibility of constructing price indexes specifically for clinical trials, an important component of healthcare R&D. Increasing health care R&D spending is an important policy issue, but changes in expenditure are very difficult to evaluate without knowing whether these increases are due to changes in the prices of inputs to biomedical research, as opposed to changes in the quantity of research being performed.

Intellectual Merit: The project constructs a "price deflator" that decomposes growth in total expenditure into price and quantity components. Very few of these price indexes have been constructed specifically for scientific research, not least because R&D activities tend to be highly heterogeneous, and therefore difficult to compare meaningfully year-to-year or across different contexts. Sufficiently detailed data make it possible both to make price comparisons that hold constant factors such as the stage of clinical development, therapeutic area, and number of patients, and to control for the complexity and work burden associated with different trial designs, The methodology that is used "hedonic price analysis" has been well-established in many areas, such as constructing deflators for IT products, but not has not previously been applied to scientific research activity. These price indexes, combined with data on clinical trial activity published in public registries such as, allow the construction of broad-based estimates of clinical trial expenditures broken out by state, country, and type of R&D performer.

Broader Impact: Along with evidence on the evolution of the price of clinical research, this project sheds light on a number of methodological issues relating to the integration of R&D investment into the national income accounts, and informs important policy debates surrounding the "outsourcing" and "offshoring" of US R&D, and the global competitiveness of American medical research establishments. Clinical trials are increasingly being performed outside traditional US academic medical centers, at physicians' offices, clinics, and provider networks, as well as by specialized for-profit entities. Moreover, trials have become increasingly global, with a single protocol collecting data from sites in many different countries. These new features of clinical trials are one example of broader changes under way in the distribution of R&D activity across geographies, actors, and institutions. This research project helps develop a better understanding of the economic forces underlying this phenomenon.

This approach can be used by BEA and other agencies in developing estimates of the impact of R&D on real and nominal gross domestic product, even at the state or regional level, which is very difficult to do with current data.



16. Human Capital and Career Mobility in Science and Engineering- Intensive Start-ups: An Open Access Initial Public Offerings Database.

Researchers: Martin Kenney, University of California Davis.

Abstract: This project extends an existing database (1996-2008) of the educational and employment histories of the entire top management team and board of directors for firms making initial public stock offering to the period from 1988-1996. The ultimate database will include more that 3,500 newly public firms and 35,000 managers and directors providing career path information for these individuals. These individuals are a central channel by which investments in science and engineering (S&E) research are transformed into economic benefits. In this way, the project addresses one of the primary goals of the Science of Science and Innovation Policy, that of evaluating and tracking the tangible but difficult to measure returns from investments in R&D.

Intellectual Merit: The database permits exploration of many areas of inquiry regarding innovation and entrepreneurship. The immediate research examines the following topics: First, because the data is drawn from official biographies, it is possible to analyze the national origin and positions immigrants occupy in high-technology startups. This is a powerful contribution, because earlier research has only studied Indian and Chinese immigrants, the data allows the identification of immigrants and provides a rigorous definition rather than one based on extrapolation from surnames. This approach allows a more scientifically valid analysis. Second, the educational background allows the identification of particular universities and departments making inordinate contributions to specific high-technology industries. This can be used to identify key institutions and programs that might provide templates for wider adoption. Third, the new data can be used to examine the degree to which these most successful high-technology firms are located in regional clusters.

Broader Impacts: The creation of a comprehensive database of the individuals involved in initial public offerings has transformative potential. The key to the success of the U.S. economy during the last two decades has been S&E-based entrepreneurship. The analysis of this database contributes to academic studies of key human capital, and, as importantly, permits the optimization of S&E policy meant to encourage small business investment and entrepreneurship. The database is made available to all interested researchers, including business school scholars, economists, geographers, and sociologists. Research based on this database should expand our understanding of the individuals responsible for many of the most successful firms in the U.S. economy



17. Scientists and Engineers as Agents of Technological Progress: Measuring the Returns to R&D and the Economic Impact of Science & Engineering Workers.

Researchers : Richard Freeman, Erling Barth and Andrew Wang, National Bureau of Economic Research and Gerald Marschke, Harvard University (cofund with Economics)

Abstract: The project develops datasets and methods to estimate the economic return to R&D spending and the impact of both R&D and non-R&D science and engineering (S&E) workers on economic outcomes, and to assess the effects of the mobility of S&E workers on economic productivity. Existing data from multiple government (Census, NSF, BLS) and other archival sources are matched and linked together to create an "Amalgamated Science and Engineering Impact" (ASEI) database. The database includes measures of output and inputs of firms and plants, R&D expenditures of firms and universities, patents of firms and other organizations, S&E employment by industry and geography, and the mobility of workers across firms and plants.

Intellectual Merit: The research connects diverse measures of S&E activity to outcomes in the economy overall and in different sectors. The project produces:

1. Updated estimates of private returns to R&D, not only in the widely studied manufacturing sector, but also in other sectors of the economy, and for small firms as well as large firms;

2. New estimates of the social returns to R&D, using measures of "spillover" R&D stock based on geographic region, industry sector, and technology proximity;

3. Estimates of the effect of own R&D and spillover R&D on patent output and market value of firms, and of the effect of patents on productivity of firms;

4. Estimates of the impact of non-R&D S&E employment on firms? productivity, patents, and other outcomes;

5. Estimates of the mobility of S&E workers across firms and plants, and of the effect of S&E worker mobility on economic outcomes;

6. Estimates of the impact of R&D on the changing demand for labor with different skill levels.

Broader Impacts: The outcomes from this research inform policymakers about the contribution of science and engineering to the U.S. economy, consistent with the goals of the NSF SciSIP program. The research provides evidence on the changing returns to R&D in the U.S., on the mobility and impact of S&E workers in the U.S. economy, and on the future demand for labor with different types of skills. The new ASEI database and analytical methods enable rapid updating of estimated results on a regular basis. This can provide policymakers with timely data and analysis on policy issues of interest.



18. Applied Visual Analytics for Economic Decision-Making.

Researchers: David Ebert, Timothy Cason, David Hummels, and Anya Savikhin, Purdue University. (cofund with CISE)

Abstract: Scientists have discovered that individuals are often unable to make optimal decisions when problems are complex due to limitations on cognitive abilities. This interdisciplinary project employs visual analytics as a transformational analytical tool in economics. The investigators use visual analytics to improve decision making and identify key motivations in knowledge creation in various economic problems. The project’s suite of tools allows users to interactively explore datasets and decision spaces as well as compare alternate hypothesis and develop new hypothesis. Further, keystrokes and information pertinent to understanding decision-making and knowledge generation are recorded, allowing the investigators to make predictions about the decision-making process on a broad scale and providing guidance for theoretical models of decision-making. This is the first thorough investigation of the value of visual analytics for economic decision-making.

Intellectual Merit: This project brings together a team of scientists from economics, electrical and computer engineering and cognitive science, fields that are rarely linked. The fundamental objective of this three-year project is to improve individual and group economic decision making through the introduction of visual analytics as a necessary tool for dealing with complex information sets. The project?s second objective is to quantify the effectiveness of visual analytics for decision making. Visual analytics has emerged as an important approach to data analysis in many fields such as medicine, business, and the physical sciences, and the investigators are the first to quantify its value for decision-making using rigorous experimental methods. The final objective is to develop a unique suite of visual analytics tools to help economists and policy-makers analyze large datasets.

Broader Impact: The use of visual analytics for economic decision-making is extremely beneficial to policy makers. Use of these tools should have an immediate and positive impact on the capacity to analyze complex economic datasets. These tools can also be used in many fields with problems in analytical reasoning. The visual analytics tools that result at the completion of the project will be made available online for classroom use, which will have a broad impact on education. Visual analytics tools are unique in that they are both simple enough and captivating for K-12 students, while also being helpful to students at the undergraduate and graduate levels. The project will use over 650 undergraduate student subjects drawn from a large and diversified student population and will provide these students with important exposure to modern research methods. The VSEEL laboratory at Purdue has an excellent record of involving members of underrepresented minority groups at both the undergraduate and graduate levels and this project is expected to continue this tradition. Over 40 percent of these student subjects will be women and about one-third will be underrepresented minorities. Based on past experience, we expect that at least one-half of the Ph.D. student researchers will be women and/or minorities.



19. A Visual Analytics Approach to Science and Innovation Policy.

Researchers: Martin Ribarsky, Remco Chang, Jim Yang, University of North Carolina, Charlotte.

Abstract: A fair amount of work, such as map of science visualization methods, has been done in the area of visualization of scientific discovery and of the relationships among scientific disciplines. Typically, both the maps and the portfolio analyses are derived from keyword and/or citation analyses of research papers coupled with categorizations by discipline of journals and conferences. Although useful, this analysis is far from complete because it does not consider the full text of the papers, just the keywords and citations, and does not consider other sources, such as research project abstracts compiled by funding agencies and reports published by agencies or research organizations. A complete analysis upon which to base policy decisions, evaluations of the effectiveness of funding, or assessments of the direction of a field would include an integrated analysis of all these sources.

Intellectual Merit: This project develops a visual analytics approach to perform these assessments of multiple sources, including full text of papers, abstracts, and reports that have not been available before. The approach is exploratory, supporting investigations where one does not initially know precisely what one is looking for but rather uses tools that permit the discovery of new relations and the uncovering of insights. Once found these insights can be looked at in more detail, tested with the gathering of new evidence, and then be the basis of further insight discovery. To support this exploratory investigation, analyses must be, at least in initial stages, unstructured and automated. The most significant words and relations must bubble up from the texts themselves. They must be automated because there will be too many text documents to assess in any other way. Yet, the analyses cannot be completely automated; there must be a place to insert understanding, to organize and make sense of what is found and to direct the investigation in a new direction based on what is found. This is exactly where interactive visual analytics makes its contribution, revealing to investigators detailed results in understandable visual displays, providing clues to prompt further exploration, and supporting organization and annotation of collected evidence and pursuit of new hypotheses. This project also looks at changes and trends over time in the paper and other collections. Detailed examination of changes and trends over time brings out behaviors that may be caused by new or newly revealed directions chosen by researchers in a field, by changes in funding, or by new and important applications. The approach that is applied is based on analyses of streaming text organized into stories, reports, or similar narrative structures. The streaming stories are organized on the fly into “event clusters” of similar stories that begin and end at particular points in time and have detailed time structures. The goal is to identify motivating events such as new funding directions, new directions established by leaders in a field, as well as new interdisciplinary thrusts across fields.

Broader Impacts: As indicated by the recent Visualization of Scientific Discovery Workshop (September, 2008), there is a significant interest and need for visualization. In addition, the just- released Science of Science Policy Roadmap (November, 2008) from the Office of Science and Technology Policy prominently mentions the need for visualization, and in particular visual analytics, tools for science analysis. This report also mentions the need for assessment of these tools for real science analysis applications. This project is positioned to meet both these needs by pursuing the development and assessment of a broad, flexible visual analytics approach for real science and innovation policy applications with real data.