Arvados

Arvados is a modern open source platform that specializes in managing and processing large biomedical data sets, including genomics and imaging data. It supports the Common Workflow Language (CWL) for writing portable and reusable workflows, and can organize and analyze petabytes of data while running reproducible and versioned computational workflows. Arvados supports the entire data life cycle, from acquisition to analysis, secure sharing, auditing, and reuse, ensuring data integrity through its content-addressable storage system called Keep, which also supports de-duplication.

Arvados uses a workflow engine to run, manage, and monitor scalable, portable, production-ready workflows, and supports running containerized workflows at scale using Docker and Singularity. It offers flexible deployment options, including on-premises, in the cloud on AWS, Azure, and GCP, as well as hybrid clusters. Arvados allows for secure data sharing and collaboration on projects while complying with data protection regulations, including HIPAA Security and Privacy standards.

Bioinformaticians use Arvados for computationally intensive data analysis and machine learning, while developers can create biomedical applications and build secure services that integrate with existing infrastructure. System administrators can run efficient and secure scalable workflows, provide flexible data management, and manage costs. Arvados is utilized by pharmaceutical companies, biotech startups, and research institutions for clinical sequencing, drug development, and diagnostic testing, helping to accelerate new drug identification and validation by running sophisticated analytical and machine learning workflows.

Arvados integrates management, analysis, and storage of biomedical image data, handling petabytes of high-resolution imaging data, and helps create biomedical data commons that aggregate and search across sequencing, omics, and imaging data hosted on different clusters. It supports federated multi-cluster workflows, allowing different steps of a workflow to execute on different clusters. Arvados is 100% free and open source software, developed in public and backed by strong commercial support, and provides a public playground for evaluation and trial use. It integrates with various enterprise authentication mechanisms, including LDAP and OpenID Connect, and uses industry-standard TLS to encrypt all data in transit.