Single Particle Analysis in Cryo-Electron Microscopy with Scipion

The CryoEM image processing use case in the EOSC Data Commons project aims to demonstrate how domain-specific scientific workflows can be seamlessly integrated into a federated cloud infrastructure. This use case focuses on deploying Scipion, a widely used open-source platform for CryoEM data analysis, on the cloud machines provided by the EOSC Data Commons.

Researchers can upload their raw CryoEM image data to the platform, where it becomes accessible for automated processing through predefined Scipion workflows. These workflows encompass the entire CryoEM analysis pipeline, including motion correction, contrast transfer function estimation, particle picking, and three-dimensional reconstruction. The orchestration and deployment of these analyses are managed using the platform’s underlying infrastructure, enabling efficient resource allocation and execution. This setup facilitates scalable and reproducible data analysis while also serving as a model for integrating advanced imaging tools into EOSC’s federated environment. The use case helps evaluate the platform’s ability to support complex, compute-intensive life science applications and provides insights into interoperability, metadata management, and automation requirements.

Scipion is a mature and robust workflow engine that already supports stream processing of cryo-EM data, enabling real-time analysis as data is acquired. Many CryoEM facilities use Scipion to perform on-the-fly tasks such as motion correction, CTF estimation, and particle picking. However, access to sufficient computational resources during acquisition remains uneven across Europe. In some countries, facilities are not co-located with adequate compute infrastructure or cannot provide sustained processing capabilities for all users.
At the same time, national-level e-infrastructures or EOSC-aligned compute services may have significant available capacity, but are not easily accessible from within facility workflows. There is a need for transparent, scalable, and automated access to these remote resources.

• Deploy Scipion on EOSC Data Commons cloud resources to enable remote CryoEM image processing.
• Automate the execution of complete CryoEM workflows, from raw image upload to 3D reconstruction.
• Evaluate the platform’s ability to support complex life science workflows in terms of interoperability, scalability, and usability.

EOSC Matchmaker
Publish tools in the EOSC Matchmaker.

EOSC Data Player
Pair data on the repository with tools from the EOSC Matchmaker service and use the EOSC Data Player service to execute Packages for Processing Datasets.

The use case is expected to result in a fully functional integration of Scipion within the EOSC Data Commons infrastructure, enabling users to process CryoEM datasets entirely in the cloud. It will demonstrate the successful automation of end-to-end image processing workflows, reducing the need for local installations and manual intervention.
Additionally, the use case will provide concrete feedback on the performance, interoperability, and usability of the platform when applied to high-throughput life science applications, thereby contributing to the refinement of EOSC services for broader scientific adoption.