Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Crypt4GH: a file format standard enabling native access to encrypted data.

ABSTRACT:

Motivation

The majority of genome analysis tools and pipelines require data to be decrypted for access. This potentially leaves sensitive genetic data exposed, either because the unencrypted data is not removed after analysis, or because the data leaves traces on the permanent storage medium.

Results

: We defined a file container specification enabling direct byte-level compatible random access to encrypted genetic data stored in community standards such as SAM/BAM/CRAM/VCF/BCF. By standardizing this format, we show how it can be added as a native file format to genomic libraries, enabling direct analysis of encrypted data without the need to create a decrypted copy.

Availability and implementation

The Crypt4GH specification can be found at: http://samtools.github.io/hts-specs/crypt4gh.pdf.

Supplementary information

Supplementary data are available at Bioinformatics online.

SUBMITTER: Senf A

PROVIDER: S-EPMC8522443 | biostudies-literature | 2021 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Crypt4GH: a file format standard enabling native access to encrypted data.

Senf Alexander A Davies Robert R Haziza Frédéric F Marshall John J Troncoso-Pastoriza Juan J Hofmann Oliver O Keane Thomas M TM

Bioinformatics (Oxford, England) 20210901 17

<h4>Motivation</h4>The majority of genome analysis tools and pipelines require data to be decrypted for access. This potentially leaves sensitive genetic data exposed, either because the unencrypted data is not removed after analysis, or because the data leaves traces on the permanent storage medium.<h4>Results</h4>: We defined a file container specification enabling direct byte-level compatible random access to encrypted genetic data stored in community standards such as SAM/BAM/CRAM/VCF/BCF. B ...[more]

PMID: 33543751

Similar Datasets

OME-NGFF: a next-generation file format for expanding bioimaging data-access strategies.

Project description:The rapid pace of innovation in biological imaging and the diversity of its applications have prevented the establishment of a community-agreed standardized data format. We propose that complementing established open formats such as OME-TIFF and HDF5 with a next-generation file format such as Zarr will satisfy the majority of use cases in bioimaging. Critically, a common metadata format used in all these vessels can deliver truly findable, accessible, interoperable and reusable bioimaging data.

| S-EPMC8648559 | biostudies-literature

A standard variation file format for human genome sequences.

Project description:Here we describe the Genome Variation Format (GVF) and the 10Gen dataset. GVF, an extension of Generic Feature Format version 3 (GFF3), is a simple tab-delimited format for DNA variant files, which uses Sequence Ontology to describe genome variation data. The 10Gen dataset, ten human genomes in GVF format, is freely available for community analysis from the Sequence Ontology website and from an Amazon elastic block storage (EBS) snapshot for use in Amazon's EC2 cloud computing environment.

| S-EPMC2945790 | biostudies-literature

ISAC's classification results file format.

Project description:Identifying homogenous sets of cell populations in flow cytometry is an important process for sorting and selecting populations of interests for further data acquisition and analysis. Many computational methods are now available to automate this process, with several algorithms partitioning cells based on high-dimensional separation versus the traditional pairwise two-dimensional visualization approach of manual gating. ISAC's classification results file format was developed to exchange the results of both manual gating and algorithmic classification approaches in a standardized way based on per event based classifications, including the potential for soft classifications expressed as the probability of an event being a member of a class. © 2014 International Society for Advancement of Cytometry.

| S-EPMC4874736 | biostudies-literature

The Biological Object Notation (BON): a structured file format for biological data.

Project description:The large size and high complexity of biological data can represent a major methodological challenge for the analysis and exchange of data sets between computers and applications. There has also been a substantial increase in the amount of metadata associated with biological data sets, which is being increasingly incorporated into existing data formats. Despite the existence of structured formats based on XML, biological data sets are mainly formatted using unstructured file formats, and the incorporation of metadata results in increasingly complex parsing routines such that they become more error prone. To overcome these problems, we present the "biological object notation" (BON) format, a new way to exchange and parse nearly all biological data sets more efficiently and with less error than other currently available formats. Based on JavaScript Object Notation (JSON), BON simplifies parsing by clearly separating the biological data from its metadata and reduces complexity compared to XML based formats. The ability to selectively compress data up to 87% compared to other file formats and the reduced complexity results in improved transfer times and less error prone applications.

| S-EPMC6018389 | biostudies-literature

Data File Standard for Flow Cytometry, version FCS 3.1.

Project description:The flow cytometry data file standard provides the specifications needed to completely describe flow cytometry data sets within the confines of the file containing the experimental data. In 1984, the first Flow Cytometry Standard format for data files was adopted as FCS 1.0. This standard was modified in 1990 as FCS 2.0 and again in 1997 as FCS 3.0. We report here on the next generation flow cytometry standard data file format. FCS 3.1 is a minor revision based on suggested improvements from the community. The unchanged goal of the standard is to provide a uniform file format that allows files created by one type of acquisition hardware and software to be analyzed by any other type.The FCS 3.1 standard retains the basic FCS file structure and most features of previous versions of the standard. Changes included in FCS 3.1 address potential ambiguities in the previous versions and provide a more robust standard. The major changes include simplified support for international characters and improved support for storing compensation. The major additions are support for preferred display scale, a standardized way of capturing the sample volume, information about originality of the data file, and support for plate and well identification in high throughput, plate based experiments. Please see the normative version of the FCS 3.1 specification in Supporting Information for this manuscript (or at http://www.isac-net.org/ in the Current standards section) for a complete list of changes.

| S-EPMC2892967 | biostudies-literature

Data File Standard for Flow Cytometry, Version FCS 3.2.

Project description:FCS 3.2 is a revision of the flow cytometry data standard based on a decade of suggested improvements from the community as well as industry needs to capture instrument conditions and measurement features more precisely. The unchanged goal of the standard is to provide a uniform file format that allows files created by one type of acquisition hardware and software to be analyzed by any other type. The standard retains the overall FCS file structure and most features of previous versions, but also contains a few changes that were required to support new types of data and use cases efficiently. These changes are incompatible with existing FCS file readers. Notably, FCS 3.2 supports mixed data types to, for example, allow FCS measurements that are intrinsically integers (e.g., indices or class assignments) or measurements that are commonly captured as integers (e.g., time ticks) to be more represented as integer values, while capturing other measurements as floating-point values in the same FCS data set. In addition, keywords explicitly specifying dyes, detectors, and analytes were added to avoid having to extract those heuristically and unreliably from measurement names. Types of measurements were formalized, several keywords added, others removed, or deprecated, and various aspects of the specification were clarified. A reference implementation of the cyclic redundancy check (CRC) calculation is provided in two programming languages since a correct CRC implementation was problematic for many vendors. © 2020 International Society for Advancement of Cytometry.

| S-EPMC8241566 | biostudies-literature

CRAM 3.1: Advances in the CRAM File Format.

Project description:CRAM has established itself as a high compression alternative to the BAM file format for DNA sequencing data. We describe updates to further improve this on modern sequencing instruments. With Illumina data CRAM 3.1 is 7 to 15% smaller than the equivalent CRAM 3.0 file, and 50 to 70% smaller than the corresponding BAM file. Long-read technology shows more modest compression due to the presence of high-entropy signals. The CRAM 3.0 specification is freely available from https://samtools.github.io/hts-specs/CRAMv3.pdf. The CRAM 3.1 improvements are available in a separate OpenSource HTScodecs library from from https://github.com/samtools/htscodecs, and have been incorporated into HTSlib. Supplementary data are available online.

| S-EPMC8896640 | biostudies-literature

NIfTI-MRS: A standard data format for magnetic resonance spectroscopy.

Project description:PurposeMultiple data formats in the MRS community currently hinder data sharing and integration. NIfTI-MRS is proposed as a standard spectroscopy data format, implemented as an extension to the Neuroimaging informatics technology initiative (NIfTI) format. This standardized format can facilitate data sharing and algorithm development as well as ease integration of MRS analysis alongside other imaging modalities.MethodsA file format using the NIfTI header extension framework incorporates essential spectroscopic metadata and additional encoding dimensions. A detailed description of the specification is provided. An open-source command-line conversion program is implemented to convert single-voxel and spectroscopic imaging data to NIfTI-MRS. Visualization of data in NIfTI-MRS is provided by development of a dedicated plugin for FSLeyes, the FMRIB Software Library (FSL) image viewer.ResultsOnline documentation and 10 example datasets in the proposed format are provided. Code examples of NIfTI-MRS readers are implemented in common programming languages. Conversion software, spec2nii, currently converts 14 formats where data is stored in image-space to NIfTI-MRS, including Digital Imaging and Communications in Medicine (DICOM) and vendor proprietary formats.ConclusionNIfTI-MRS aims to solve issues arising from multiple data formats being used in the MRS community. Through a single conversion point, processing and analysis of MRS data are simplified, thereby lowering the barrier to use of MRS. Furthermore, it can serve as the basis for open data sharing, collaboration, and interoperability of analysis programs. Greater standardization and harmonization become possible. By aligning with the dominant format in neuroimaging, NIfTI-MRS enables the use of mature tools present in the imaging community, demonstrated in this work by using a dedicated imaging tool, FSLeyes, for visualization.

| S-EPMC7613677 | biostudies-literature

Interpretation errors related to the GO annotation file format.

Project description:The Gene Ontology (GO) is the most widely used ontology for creating biomedical annotations. GO annotations are statements associating a biological entity with a GO term. These statements comprise a large dataset of biological knowledge that is used widely in biomedical research. GO Annotations are available as "gene association files" from the GO website in a tab-delimited file format (GO Annotation File Format) composed of rows of 15 tab-delimited fields. This simple format lacks the knowledge representation (KR) capabilities to represent unambiguously semantic relationships between each field. This paper demonstrates that this KR shortcoming leads users to interpret the files in ways that can be erroneous. We propose a complementary format to represent GO annotation files as knowledge bases using the W3C recommended Web Ontology Language (OWL).

| S-EPMC2655813 | biostudies-literature

Assessing and assuring interoperability of a genomics file format.

Project description:MotivationBioinformatics software tools operate largely through the use of specialized genomics file formats. Often these formats lack formal specification, making it difficult or impossible for the creators of these tools to robustly test them for correct handling of input and output. This causes problems in interoperability between different tools that, at best, wastes time and frustrates users. At worst, interoperability issues could lead to undetected errors in scientific results.ResultsWe developed a new verification system, Acidbio, which tests for correct behavior in bioinformatics software packages. We crafted tests to unify correct behavior when tools encounter various edge cases-potentially unexpected inputs that exemplify the limits of the format. To analyze the performance of existing software, we tested the input validation of 80 Bioconda packages that parsed the Browser Extensible Data (BED) format. We also used a fuzzing approach to automatically perform additional testing. Of 80 software packages examined, 75 achieved less than 70% correctness on our test suite. We categorized multiple root causes for the poor performance of different types of software. Fuzzing detected other errors that the manually designed test suite could not. We also created a badge system that developers can use to indicate more precisely which BED variants their software accepts and to advertise the software's performance on the test suite.Availability and implementationAcidbio is available at https://github.com/hoffmangroup/acidbio.Supplementary informationSupplementary data are available at Bioinformatics online.

| S-EPMC9237710 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data