Data processing

“ When doing this, one will frequently find that getting from raw data to the final result involves many interrelated steps (single commands, scripts, programs) . We refer to such a sequence of steps, whether it is automated or performed manually, as an analysis workflow. While the essential part of an analysis is often represented by only one of the steps, the full sequence of pre- and post-processing steps are often critical in order to reach the achieved result. For every involved step, you should ensure that every detail that may influence the execution of the step is recorded. ”

“ We expect the competitive environment within which data processing services are now being offered to result in substantial public benefit by making available to the public, at reasonable charges, a wider range of existing and new data processing services. We believe that these expectations will continue to be realized in the free give-and-take of the market place without the need for and possible burden of rules, regulations and licensing requirements. ”

“ The implication here is of the computer not merely as a data-handler, but as a business organizer and planner as well.
Electronic Ticker Tape The stock market lends itself to the use of high-speed data-processing, even though a Wall Street man achieved notoriety some time back as the first embezzler to use computer techniques. ”

“ Computer technology is increasingly removing technical limitations as to the types of enhanced services that may be offered. Yet, a classification scheme which would categorize enhanced services as either communications or data processing inherently limits the types of services that an unregulated entity may offer. The reason for this is clear. Providers of data processing and other computer services acquire the necessary transmission facilities from communications common carriers pursuant to tariff, and resell this transmission capability as part of their enhanced offering. ”

“ Statistics on family in the USA and the world are a matter of public record. The processing and interpretation of data, even in the age of electronic processing, takes time once data has been collected. ”

“ Introduction Data is the central currency of science, but the nature of scientific data has changed dramatically with the rapid pace of technology. This change has led to the development of a wide variety of data formats, dataset sizes, data complexity, data use cases, and data sharing practices. ”

“ Future work Future work could improve on these results by considering and integrating all methods of data use attribution. This holistic effort would include identifying citations to the paper that describes the data collection, mentions of the dataset identifier itself – whether in full text, the references section, or supplementary information – citations to the dataset as a first-class research object, and even mentions of the data collection investigators in acknowledgement sections. ”

“ Rule 1. Love Your Data, and Help Others Love It, Too Data management is a repeat-play game. If you take care to make your data easily available to others, others are more likely to do the same—eventually. ”

“ Experienced reviewers will be on the lookout for unfunded components, but they will also recognize that greater or lesser investments in data management depend upon the nature of the research and the types of data. ”

“ Such plans typically cover all or portions of the data life cycle—from data discovery, collection, and organization (e.g., spreadsheets, databases) , through quality assurance/quality control, documentation (e.g., data types, laboratory methods) and use of the data, to data preservation and sharing with others (e.g., data policies and dissemination approaches) . Fig 1 illustrates the relationship between hypothetical research and data life cycles and highlights the links to the rules presented in this paper. The DMP undergoes peer review and is used in part to evaluate a project’s merit. ”

“ Data papers should describe in detail how the data was acquired, with which goals and constraints, an assessment of their quality, how they have and how they can or should be reused, how to get access, give feedback and credit. Datasets are technical and critical building blocks of science and should be recognized as such by high impact and heavy citation, ensuring that creators of data are appropriately acknowledged for their work. ”

“ The attribution system used for scholarly articles, accomplished via citations, often breaks in the case of data and software. When other authors reuse or cite your data or code, you may get an acknowledgment or an incoming link. If you and your colleagues have gone to the trouble to write a “data paper,” whose main purpose is to describe your data and/or code, you may also get a citation [8] . But, “data paper” writing is not always desirable, or relevant. So, how do you go about getting the full credit you deserve for your data and code? ”

“ We have refrained from using this concept throughout this discussion, but at this point it is appropriate to recognize that some informatics literature describes the informatics continuum (earlier referred to as the data progression) as data to information to knowledge to wisdom. [26] Typically, finding objective measures of wisdom is not easy or universally accepted. However, we have chosen to substitute wisdom for decision and outcomes. As a result, our data progression extends to the following sequence: data to information to knowledge to decisions to outcomes. ”

“ Traditionally, transmission capacity has been offered for discrete services, such as telephone service. With the incorporation of digital technology into the telephone network and the inclusion of computer processing capabilities into both terminal equipment located in the customer's premises and the equipment making up a firm's "network," this is no longer the case. Telecommunications service is no longer just "plain old telephone service" to the user. A subscriber may use telephone service to transmit voice or data. ”

“ Importantly, almost all of these mechanisms are aligned with more efficient and effective scientific progress: increased data use, facilitated credibility determination, earlier access, improved discoverability, and a focus on best work through data availability are good for both investigators and the science community as a whole. ”

“ Don’t reinvent the wheel Creating a high-quality database is a responsibility that involves strong commitment to accurate data collection and regular content and feature updates, not to mention a substantial time investment. ”

“ Using the tidy format makes this distinction clearer. One axiom about the structure of data and code holds that one should “write code for humans, write data for computers” [ [23] ] . When data can be easily imported and manipulated using familiar software (whether via a scripting language, a spreadsheet, or any other computer program that can import these common files) , data becomes easier to re-use. Furthermore, having the source code for the software doing the analysis available provides provenance for how the data is processed and analyzed. ”

“ Perfect documentation of provenance is rarely, if ever, attained in scientific work today. The higher the quality of provenance information, the higher the chance of enabling data reuse. In general, data reuse is most possible when: 1) data; 2) metadata (information describing the data) ; and 3) information about the process of generating those data, such as code, are all provided. ”

“ We seek to improve on prior work in two key ways. First, the sample size of this analysis is large – over two orders of magnitude larger than the first citation study of gene expression microarray data (Piwowar, Day & Fridsma, 2007) , giving us the statistical power to account for a larger number of cofactors in the analyses. Thus, the resulting estimates isolate the association between data availability and citation rate with more accuracy. Second, this report goes beyond citation analysis to include analysis of data reuse attribution directly. ”

“ Generate Hierarchical Analysis Output, Allowing Layers of Increasing Detail to Be Inspected The final results that make it to an article, be it plots or tables, often represent highly summarized data. For instance, each value along a curve may in turn represent averages from an underlying distribution. In order to validate and fully understand the main result, it is often useful to inspect the detailed values underlying the summaries. A common but impractical way of doing this is to incorporate various debug outputs in the source code of scripts and programs. ”

“ The strongest reason to create a new database is scientific demand for a type of data not easily available in a computable form anywhere else. It is most useful to have all data of a single type in one easy-to-search location, so, ideally, everyone interested in collecting data about a specific topic should collaborate to create one resource, or at least should coordinate efforts to reduce duplication of work (Fig 1) . Either way, the data content and software features you create will have the greatest impact if they are original and useful ”

“ Introduction Biological databases are online libraries that contain structured information about living organisms. These databases are indispensable research tools, as they provide convenient, computable access to prior knowledge that is vital for planning future experiments and for discovering new knowledge through data mining—they help us “stand on the shoulders of giants.” ”

“ We've been making statistical reports on society for two centuries. We have immense files of data. The machines are able to predict what each person and group will do at a given time, in a given situation. ”

“ Asking questions at the design stage can save headaches at the analysis stage: careful data collection can greatly simplify analysis and make it more rigorous. Or, as Sir Ronald Fisher put it: “To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. ”

“ Even as identifier standards may change over time, datasets can evolve over time as well. In order to distinguish between different versions of the same data, each dataset should have a distinct name, which includes a version identifier. ”

“ Databases with a good user experience make it easy to navigate and find data even by non-experts. Database navigation is facilitated by a graphical user interface that organizes, presents, and visualizes these data in a human accessible manner. ”

“ Our understanding of research practice suffers because access to tools and data is so difficult.
Patterns of data reuse To better understand patterns of data reuse, a larger sample of reuse instances was needed than could easily be assembled through manual classification of citation context. To that end, we used a complementary source of information about reuse of the same datasets: direct mention of GEO or ArrayExpress accession numbers within the body of a full-text research article. ”

“ With more experienced scientists as biocurators, an accurate and controlled vocabulary among database users and contributors can be developed and promoted. The use of a shared vocabulary increases curation speed, data consistency, and serves to connect the information contained in multiple databases. ”

“ Note that many research sponsors define data broadly to include physical collections, software and code, and curriculum materials.# Sources. Data may come from direct human observation, laboratory and field instruments, experiments, simulations, and compilations of data from other studies. Reviewers and sponsors may be particularly interested in understanding if data are proprietary, are being compiled from other studies, pertain to human subjects, or are otherwise subject to restrictions in their use or redistribution.# Volume. ”

“ Research papers and data products are key outcomes of the science enterprise. Governmental, nongovernmental, and private foundation sponsors of research are increasingly recognizing the value of research data. As a result, most funders now require that sufficiently detailed data management plans be submitted as part of a research proposal. A data management plan (DMP) is a document that describes how you will treat your data during a project and what happens with the data after the project ends. ”

“ Among the interesting issues we have not dealt with or have not yet implemented' in RABBIT are: compilation of data manipulation primitives, interaction of such primitives, procedure integration of the most general form, and complex register allocation. A particularly interesting issue is that of data type analysis. ”

“ Passive approaches include posting data on a project or personal website or mailing or emailing data upon request, although the latter can be problematic when dealing with large data and bandwidth constraints. More active, robust, and preferred approaches include: (1) publishing the data in an open repository or archive (see Rule 6) ; (2) submitting the data (or subsets thereof) as appendices or supplements to journal articles, such as is commonly done with the PLOS family of journals; and (3) publishing the data, metadata, and relevant code as a “data paper” [5] . ”

“ Creating, managing, publishing, and sharing high-quality data is as much a part of the 21st century research enterprise as is publishing the results. Data management is not new—rather, it is something that all researchers already do. Nonetheless, a common mistake in developing a DMP is forgetting to budget for the activities. Data management takes time and costs money in terms of software, hardware, and personnel. ”

“ Be a Booster for Data Science As Rule 1 says, it is important not just that you love your own data, but that others love data, too. An attitude that data and code are “second-class objects,” behind traditional scholarly publications, is still prevalent. ”